Sarah A. Thomas

Project 6 - Thera Bank

Description (copied from assignment)

Background & Context

The Thera bank recently saw a steep decline in the number of users of their credit card, credit cards are a good source of income for banks because of different kinds of fees charged by the banks like annual fees, balance transfer fees, and cash advance fees, late payment fees, foreign transaction fees, and others. Some fees are charged to every user irrespective of usage, while others are charged under specified circumstances.

Customers’ leaving credit cards services would lead bank to loss, so the bank wants to analyze the data of customers and identify the customers who will leave their credit card services and reason for same – so that bank could improve upon those areas

You as a Data scientist at Thera bank need to come up with a classification model that will help the bank improve its services so that customers do not renounce their credit cards

You need to identify the best possible model that will give the required performance

Objective

1. Explore and visualize the dataset.
2. Build a classification model to predict if the customer is going to churn or not
3. Optimize the model using appropriate techniques
4. Generate a set of insights and recommendations that will help the bank

Data Dictionary:

• CLIENTNUM: Client number. Unique identifier for the customer holding the account
• Attrition_Flag: Internal event (customer activity) variable - if the account is closed then "Attrited Customer" else "Existing Customer"
• Customer_Age: Age in Years
• Gender: Gender of the account holder
• Dependent_count: Number of dependents
• Education_Level: Educational Qualification of the account holder - Graduate, High School, Unknown, Uneducated, College(refers to a college student), Post-Graduate, Doctorate.
• Marital_Status: Marital Status of the account holder
• Income_Category: Annual Income Category of the account holder
• Card_Category: Type of Card
• Months_on_book: Period of relationship with the bank
• Total_Relationship_Count: Total no. of products held by the customer
• Months_Inactive_12_mon: No. of months inactive in the last 12 months
• Contacts_Count_12_mon: No. of Contacts between the customer and bank in the last 12 months
• Credit_Limit: Credit Limit on the Credit Card
• Total_Revolving_Bal: The balance that carries over from one month to the next is the revolving balance
• Avg_Open_To_Buy: Open to Buy refers to the amount left on the credit card to use (Average of last 12 months)
• Total_Trans_Amt: Total Transaction Amount (Last 12 months)
• Total_Trans_Ct: Total Transaction Count (Last 12 months)
• Total_Ct_Chng_Q4_Q1: Ratio of the total transaction count in 4th quarter and the total transaction count in 1st quarter
• Total_Amt_Chng_Q4_Q1: Ratio of the total transaction amount in 4th quarter and the total transaction amount in 1st quarter
• Avg_Utilization_Ratio: Represents how much of the available credit the customer spent

1 - Load Packages and Read in the Dataset

# Library to suppress warnings or deprecation notes
import warnings

warnings.filterwarnings("ignore")

# Import time to check the training time
import time
import datetime

# Libraries to help with reading and manipulating data
import numpy as np
import pandas as pd

# Libraries to help with data visualization
import matplotlib.pyplot as plt

%matplotlib inline
import seaborn as sns

# Libraries to split data, impute missing values
from sklearn.model_selection import train_test_split
from sklearn.impute import SimpleImputer

# To help with model building
from sklearn.linear_model import LogisticRegression

# Libraries to import decision tree classifier and different ensemble classifiers
from sklearn.ensemble import BaggingClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import AdaBoostClassifier, GradientBoostingClassifier
from xgboost import XGBClassifier
from sklearn.ensemble import StackingClassifier
from sklearn.tree import DecisionTreeClassifier

# Libtune to tune model, get different metric scores
from sklearn import metrics
from sklearn.metrics import (
    confusion_matrix,
    classification_report,
    accuracy_score,
    precision_score,
    recall_score,
    f1_score,
    roc_auc_score,
    plot_confusion_matrix,
)

# To be used for data scaling and one hot encoding
from sklearn.preprocessing import StandardScaler, MinMaxScaler, OneHotEncoder

# To be used for tuning the model
from sklearn.model_selection import (
    GridSearchCV,
    RandomizedSearchCV,
    StratifiedKFold,
    cross_val_score,
)

# To be used for creating pipelines and personalizing them
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer

# To oversample and undersample data
from imblearn.over_sampling import SMOTE
from imblearn.under_sampling import RandomUnderSampler

# to make Python code more structured
%load_ext nb_black

# To supress scientific notations for a dataframe
pd.set_option("display.float_format", lambda x: "%.3f" % x)

# Remove limit from displayed columns and rows.
pd.set_option("display.max_columns", None)
pd.set_option("display.max_rows", 200)

# load the data into pandas dataframe
bank = pd.read_csv("BankChurners.csv")

2 - Data Preprocessing

2.1 - Check the first and last 10 rows of the dataset and random 10 rows

bank.head(10)

	CLIENTNUM	Attrition_Flag	Customer_Age	Gender	Dependent_count	Education_Level	Marital_Status	Income_Category	Card_Category	Months_on_book	Total_Relationship_Count	Months_Inactive_12_mon	Contacts_Count_12_mon	Credit_Limit	Total_Revolving_Bal	Avg_Open_To_Buy	Total_Amt_Chng_Q4_Q1	Total_Trans_Amt	Total_Trans_Ct	Total_Ct_Chng_Q4_Q1	Avg_Utilization_Ratio
0	768805383	Existing Customer	45	M	3	High School	Married	$60K - $80K	Blue	39	5	1	3	12691.000	777	11914.000	1.335	1144	42	1.625	0.061
1	818770008	Existing Customer	49	F	5	Graduate	Single	Less than $40K	Blue	44	6	1	2	8256.000	864	7392.000	1.541	1291	33	3.714	0.105
2	713982108	Existing Customer	51	M	3	Graduate	Married	$80K - $120K	Blue	36	4	1	0	3418.000	0	3418.000	2.594	1887	20	2.333	0.000
3	769911858	Existing Customer	40	F	4	High School	NaN	Less than $40K	Blue	34	3	4	1	3313.000	2517	796.000	1.405	1171	20	2.333	0.760
4	709106358	Existing Customer	40	M	3	Uneducated	Married	$60K - $80K	Blue	21	5	1	0	4716.000	0	4716.000	2.175	816	28	2.500	0.000
5	713061558	Existing Customer	44	M	2	Graduate	Married	$40K - $60K	Blue	36	3	1	2	4010.000	1247	2763.000	1.376	1088	24	0.846	0.311
6	810347208	Existing Customer	51	M	4	NaN	Married	$120K +	Gold	46	6	1	3	34516.000	2264	32252.000	1.975	1330	31	0.722	0.066
7	818906208	Existing Customer	32	M	0	High School	NaN	$60K - $80K	Silver	27	2	2	2	29081.000	1396	27685.000	2.204	1538	36	0.714	0.048
8	710930508	Existing Customer	37	M	3	Uneducated	Single	$60K - $80K	Blue	36	5	2	0	22352.000	2517	19835.000	3.355	1350	24	1.182	0.113
9	719661558	Existing Customer	48	M	2	Graduate	Single	$80K - $120K	Blue	36	6	3	3	11656.000	1677	9979.000	1.524	1441	32	0.882	0.144

bank.tail(10)

	CLIENTNUM	Attrition_Flag	Customer_Age	Gender	Dependent_count	Education_Level	Marital_Status	Income_Category	Card_Category	Months_on_book	Total_Relationship_Count	Months_Inactive_12_mon	Contacts_Count_12_mon	Credit_Limit	Total_Revolving_Bal	Avg_Open_To_Buy	Total_Amt_Chng_Q4_Q1	Total_Trans_Amt	Total_Trans_Ct	Total_Ct_Chng_Q4_Q1	Avg_Utilization_Ratio
10117	712503408	Existing Customer	57	M	2	Graduate	Married	$80K - $120K	Blue	40	6	3	4	17925.000	1909	16016.000	0.712	17498	111	0.820	0.106
10118	713755458	Attrited Customer	50	M	1	NaN	NaN	$80K - $120K	Blue	36	6	3	4	9959.000	952	9007.000	0.825	10310	63	1.100	0.096
10119	716893683	Attrited Customer	55	F	3	Uneducated	Single	abc	Blue	47	4	3	3	14657.000	2517	12140.000	0.166	6009	53	0.514	0.172
10120	710841183	Existing Customer	54	M	1	High School	Single	$60K - $80K	Blue	34	5	2	0	13940.000	2109	11831.000	0.660	15577	114	0.754	0.151
10121	713899383	Existing Customer	56	F	1	Graduate	Single	Less than $40K	Blue	50	4	1	4	3688.000	606	3082.000	0.570	14596	120	0.791	0.164
10122	772366833	Existing Customer	50	M	2	Graduate	Single	$40K - $60K	Blue	40	3	2	3	4003.000	1851	2152.000	0.703	15476	117	0.857	0.462
10123	710638233	Attrited Customer	41	M	2	NaN	Divorced	$40K - $60K	Blue	25	4	2	3	4277.000	2186	2091.000	0.804	8764	69	0.683	0.511
10124	716506083	Attrited Customer	44	F	1	High School	Married	Less than $40K	Blue	36	5	3	4	5409.000	0	5409.000	0.819	10291	60	0.818	0.000
10125	717406983	Attrited Customer	30	M	2	Graduate	NaN	$40K - $60K	Blue	36	4	3	3	5281.000	0	5281.000	0.535	8395	62	0.722	0.000
10126	714337233	Attrited Customer	43	F	2	Graduate	Married	Less than $40K	Silver	25	6	2	4	10388.000	1961	8427.000	0.703	10294	61	0.649	0.189

np.random.seed(1)
bank.sample(n=10)

	CLIENTNUM	Attrition_Flag	Customer_Age	Gender	Dependent_count	Education_Level	Marital_Status	Income_Category	Card_Category	Months_on_book	Total_Relationship_Count	Months_Inactive_12_mon	Contacts_Count_12_mon	Credit_Limit	Total_Revolving_Bal	Avg_Open_To_Buy	Total_Amt_Chng_Q4_Q1	Total_Trans_Amt	Total_Trans_Ct	Total_Ct_Chng_Q4_Q1	Avg_Utilization_Ratio
6498	712389108	Existing Customer	43	F	2	Graduate	Married	Less than $40K	Blue	36	6	3	2	2570.000	2107	463.000	0.651	4058	83	0.766	0.820
9013	718388733	Existing Customer	38	F	1	College	NaN	Less than $40K	Blue	32	2	3	3	2609.000	1259	1350.000	0.871	8677	96	0.627	0.483
2053	710109633	Existing Customer	39	M	2	College	Married	$60K - $80K	Blue	31	6	3	2	9871.000	1061	8810.000	0.545	1683	34	0.478	0.107
3211	717331758	Existing Customer	44	M	4	Graduate	Married	$120K +	Blue	32	6	3	4	34516.000	2517	31999.000	0.765	4228	83	0.596	0.073
5559	709460883	Attrited Customer	38	F	2	Doctorate	Married	Less than $40K	Blue	28	5	2	4	1614.000	0	1614.000	0.609	2437	46	0.438	0.000
6106	789105183	Existing Customer	54	M	3	Post-Graduate	Single	$80K - $120K	Silver	42	3	1	2	34516.000	2488	32028.000	0.552	4401	87	0.776	0.072
4150	771342183	Attrited Customer	53	F	3	Graduate	Single	$40K - $60K	Blue	40	6	3	2	1625.000	0	1625.000	0.689	2314	43	0.433	0.000
2205	708174708	Existing Customer	38	M	4	Graduate	Married	$40K - $60K	Blue	27	6	2	4	5535.000	1276	4259.000	0.636	1764	38	0.900	0.231
4145	718076733	Existing Customer	43	M	1	Graduate	Single	$60K - $80K	Silver	31	4	3	3	25824.000	1170	24654.000	0.684	3101	73	0.780	0.045
5324	821889858	Attrited Customer	50	F	1	Doctorate	Single	abc	Blue	46	6	4	3	1970.000	1477	493.000	0.662	2493	44	0.571	0.750

2.2 - Check the shape of the data

bank.shape

(10127, 21)

2.3 - Check the datatypes and rows counts for each column

bank.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 10127 entries, 0 to 10126
Data columns (total 21 columns):
 #   Column                    Non-Null Count  Dtype  
---  ------                    --------------  -----  
 0   CLIENTNUM                 10127 non-null  int64  
 1   Attrition_Flag            10127 non-null  object 
 2   Customer_Age              10127 non-null  int64  
 3   Gender                    10127 non-null  object 
 4   Dependent_count           10127 non-null  int64  
 5   Education_Level           8608 non-null   object 
 6   Marital_Status            9378 non-null   object 
 7   Income_Category           10127 non-null  object 
 8   Card_Category             10127 non-null  object 
 9   Months_on_book            10127 non-null  int64  
 10  Total_Relationship_Count  10127 non-null  int64  
 11  Months_Inactive_12_mon    10127 non-null  int64  
 12  Contacts_Count_12_mon     10127 non-null  int64  
 13  Credit_Limit              10127 non-null  float64
 14  Total_Revolving_Bal       10127 non-null  int64  
 15  Avg_Open_To_Buy           10127 non-null  float64
 16  Total_Amt_Chng_Q4_Q1      10127 non-null  float64
 17  Total_Trans_Amt           10127 non-null  int64  
 18  Total_Trans_Ct            10127 non-null  int64  
 19  Total_Ct_Chng_Q4_Q1       10127 non-null  float64
 20  Avg_Utilization_Ratio     10127 non-null  float64
dtypes: float64(5), int64(10), object(6)
memory usage: 1.6+ MB

Observations:

• After checking for duplicates, will drop CLIENTNUM since it does not provide any relevant information.
• The following have null values that need to be dealt with: Education_Level, Marital_Status.
• The following need to be converted to categorical variables: Attrition_Flag, Gender, Education_Level, Marital_Status, Income_Category, Card_Category.

2.4 - Check for duplicates and drop CLIENTNUM

bank.duplicated().sum()

0

# Used CLIENTNUM to make sure there were no duplicate records
# Since there are no duplicate records, I can now drop CLIENTNUM
bank.drop(["CLIENTNUM"], axis=1, inplace=True)

## 2.5 - Check values for soon-to-be categorical variables to ensure nothing needs correction

bank["Attrition_Flag"].value_counts()

Existing Customer    8500
Attrited Customer    1627
Name: Attrition_Flag, dtype: int64

bank["Gender"].value_counts()

F    5358
M    4769
Name: Gender, dtype: int64

bank["Education_Level"].value_counts()

Graduate         3128
High School      2013
Uneducated       1487
College          1013
Post-Graduate     516
Doctorate         451
Name: Education_Level, dtype: int64

bank["Marital_Status"].value_counts()

Married     4687
Single      3943
Divorced     748
Name: Marital_Status, dtype: int64

bank["Income_Category"].value_counts()

Less than $40K    3561
$40K - $60K       1790
$80K - $120K      1535
$60K - $80K       1402
abc               1112
$120K +            727
Name: Income_Category, dtype: int64

# abc seems to be filler for missing values - will replace with "missing"
bank["Income_Category"] = np.where(
    (bank.Income_Category == "abc"), "missing", bank.Income_Category
)
bank["Income_Category"].value_counts()

Less than $40K    3561
$40K - $60K       1790
$80K - $120K      1535
$60K - $80K       1402
missing           1112
$120K +            727
Name: Income_Category, dtype: int64

bank["Card_Category"].value_counts()

Blue        9436
Silver       555
Gold         116
Platinum      20
Name: Card_Category, dtype: int64

2.5 - Convert Categoricals

bank["Attrition_Flag"] = bank["Attrition_Flag"].astype("category")
bank["Gender"] = bank["Gender"].astype("category")
bank["Education_Level"] = bank["Education_Level"].astype("category")
bank["Marital_Status"] = bank["Marital_Status"].astype("category")
bank["Income_Category"] = bank["Income_Category"].astype("category")
bank["Card_Category"] = bank["Card_Category"].astype("category")

3 - EDA

3.1 - Explore Summary of Data

bank.describe(include="all").T

	count	unique	top	freq	mean	std	min	25%	50%	75%	max
Attrition_Flag	10127	2	Existing Customer	8500	NaN	NaN	NaN	NaN	NaN	NaN	NaN
Customer_Age	10127.000	NaN	NaN	NaN	46.326	8.017	26.000	41.000	46.000	52.000	73.000
Gender	10127	2	F	5358	NaN	NaN	NaN	NaN	NaN	NaN	NaN
Dependent_count	10127.000	NaN	NaN	NaN	2.346	1.299	0.000	1.000	2.000	3.000	5.000
Education_Level	8608	6	Graduate	3128	NaN	NaN	NaN	NaN	NaN	NaN	NaN
Marital_Status	9378	3	Married	4687	NaN	NaN	NaN	NaN	NaN	NaN	NaN
Income_Category	10127	6	Less than $40K	3561	NaN	NaN	NaN	NaN	NaN	NaN	NaN
Card_Category	10127	4	Blue	9436	NaN	NaN	NaN	NaN	NaN	NaN	NaN
Months_on_book	10127.000	NaN	NaN	NaN	35.928	7.986	13.000	31.000	36.000	40.000	56.000
Total_Relationship_Count	10127.000	NaN	NaN	NaN	3.813	1.554	1.000	3.000	4.000	5.000	6.000
Months_Inactive_12_mon	10127.000	NaN	NaN	NaN	2.341	1.011	0.000	2.000	2.000	3.000	6.000
Contacts_Count_12_mon	10127.000	NaN	NaN	NaN	2.455	1.106	0.000	2.000	2.000	3.000	6.000
Credit_Limit	10127.000	NaN	NaN	NaN	8631.954	9088.777	1438.300	2555.000	4549.000	11067.500	34516.000
Total_Revolving_Bal	10127.000	NaN	NaN	NaN	1162.814	814.987	0.000	359.000	1276.000	1784.000	2517.000
Avg_Open_To_Buy	10127.000	NaN	NaN	NaN	7469.140	9090.685	3.000	1324.500	3474.000	9859.000	34516.000
Total_Amt_Chng_Q4_Q1	10127.000	NaN	NaN	NaN	0.760	0.219	0.000	0.631	0.736	0.859	3.397
Total_Trans_Amt	10127.000	NaN	NaN	NaN	4404.086	3397.129	510.000	2155.500	3899.000	4741.000	18484.000
Total_Trans_Ct	10127.000	NaN	NaN	NaN	64.859	23.473	10.000	45.000	67.000	81.000	139.000
Total_Ct_Chng_Q4_Q1	10127.000	NaN	NaN	NaN	0.712	0.238	0.000	0.582	0.702	0.818	3.714
Avg_Utilization_Ratio	10127.000	NaN	NaN	NaN	0.275	0.276	0.000	0.023	0.176	0.503	0.999

Observations:

• Attrition_Flag: Most are existing customers.
• Customer_Age: Average age is approx. 46 (median - 46, min - 26, max - 73).
• Gender: Most customers are female.
• Dependent_count: Average is approx. 2 (median - 2, min - 0, max - 5).
• Education_Level: Most customers hold a Graduate degree.
• Marital_Status: Most customers are married.
• Income_Category: Most customers make less than \$40,000.
• Card_Category: Most customers are in the Blue card category.
• Months_on_book: The average is 36 months in relationship with bank (median - 36, min - 13, max - 56).
• Total_Relationship_Count: Customers on average have approx. 4 products with bank (median - 4, min - 1, max - 6).
• Months_Inactive_12_mon: On average, number of months inactive in the past year is approx. 2 (median - 2, min - 0, max - 6).
• Contacts_Count_12_mon: Average number of contacts with bank is approx. 2 (median - 2, min - 0, max - 6).
• Credit_Limit: Average credit limit is \$8631.95 (median - 4549.00, min - 1438.30, max - 34516.00)
• Total_Revolving_Bal: Average revolving balance is \$1162.81 (median - 1276.00, min - 0, max - 2517.00)
• Avg_Open_To_Buy: Average amount left on the credit card to use is \$7469.14 (median - 3474.00, min - 3.00, max - 34516.00)
• Total_Amt_Chng_Q4_Q1: Average is 0.76 (median - 0.74, min - 0, max - 3.40)
• Total_Trans_Amt: Average total transaction amount is \$4404.09 (median - 3899.00, min - 510.00, max - 18.484.00)
• Total_Trans_Ct: Average total transaction count is 64.86 (median - 67, min - 10, max - 139)
• Total_Ct_Chng_Q4_Q1: Average is 0.71 (median - 0.70, min - 0, max - 3.71)
• Avg_Utilization_Ratio: Average ratio is 0.28 (median - 0.18, min - 0, max - 1.00)

3.2 - Univariate Analysis

# Function copied from prior case study:
# Let us write a function that will help us create boxplot and histogram for any input numerical variable.
# This function takes the numerical column as the input and returns the boxplots and histograms for the variable.
def histogram_boxplot(feature, figsize=(15, 10), bins=None):
    """Boxplot and histogram combined
    feature: 1-d feature array
    figsize: size of fig (default (9,8))
    bins: number of bins (default None / auto)
    """
    sns.set(font_scale=2)  # setting the font scale for seaborn
    f2, (ax_box2, ax_hist2) = plt.subplots(
        nrows=2,  # Number of rows of the subplot grid=2
        sharex=True,  # x-axis will be shared among all subplots
        gridspec_kw={"height_ratios": (0.25, 0.75)},
        figsize=figsize,
    )  # creating the 2 subplots
    sns.boxplot(
        feature, ax=ax_box2, showmeans=True, color="red"
    )  # boxplot will be created and a star will indicate the mean value of the column
    sns.distplot(feature, kde=F, ax=ax_hist2, bins=bins) if bins else sns.distplot(
        feature, kde=False, ax=ax_hist2
    )  # For histogram
    ax_hist2.axvline(
        feature.mean(), color="g", linestyle="--"
    )  # Add mean to the histogram
    ax_hist2.axvline(
        feature.median(), color="black", linestyle="-"
    )  # Add median to the histogram

3.2.1 - Customer_Age

histogram_boxplot(bank["Customer_Age"])

• There are outliers so distribution is slightly right-skewed.

# Checking to see how many outliers exist
bank[bank["Customer_Age"] > 65]

	Attrition_Flag	Customer_Age	Gender	Dependent_count	Education_Level	Marital_Status	Income_Category	Card_Category	Months_on_book	Total_Relationship_Count	Months_Inactive_12_mon	Contacts_Count_12_mon	Credit_Limit	Total_Revolving_Bal	Avg_Open_To_Buy	Total_Amt_Chng_Q4_Q1	Total_Trans_Amt	Total_Trans_Ct	Total_Ct_Chng_Q4_Q1	Avg_Utilization_Ratio
39	Attrited Customer	66	F	0	Doctorate	Married	missing	Blue	56	5	4	3	7882.000	605	7277.000	1.052	704	16	0.143	0.077
52	Existing Customer	66	F	0	High School	Married	Less than $40K	Blue	54	3	4	2	3171.000	2179	992.000	1.224	1946	38	1.923	0.687
151	Existing Customer	68	M	1	Graduate	Married	missing	Blue	56	5	2	3	13860.000	1652	12208.000	1.255	1910	32	1.909	0.119
177	Existing Customer	67	F	1	Graduate	Married	Less than $40K	Blue	56	4	3	2	3006.000	2517	489.000	2.053	1661	32	1.000	0.837
198	Attrited Customer	68	M	0	High School	Married	Less than $40K	Blue	52	1	3	2	1438.300	900	538.300	0.743	760	21	0.615	0.626
251	Existing Customer	73	M	0	High School	Married	$40K - $60K	Blue	36	5	3	2	4469.000	1125	3344.000	1.363	1765	34	1.615	0.252
254	Existing Customer	70	M	0	High School	Married	Less than $40K	Blue	56	3	2	3	3252.000	1495	1757.000	0.581	1227	15	0.875	0.460
320	Existing Customer	67	M	0	Graduate	Married	$40K - $60K	Blue	56	4	2	1	5876.000	1297	4579.000	1.012	2133	55	0.719	0.221
410	Existing Customer	67	M	1	Uneducated	Married	$40K - $60K	Blue	49	4	3	2	3106.000	1675	1431.000	0.781	1512	38	0.407	0.539
432	Existing Customer	67	F	0	NaN	Married	missing	Blue	56	4	3	3	10509.000	2388	8121.000	0.777	1365	34	1.429	0.227

• So few outliers exist and they exist so close to the other values that they need not be treated.

3.2.2 - Dependent_count

histogram_boxplot(bank["Dependent_count"])

• Distribution is symmetrical.

3.2.3 - Months_on_book

histogram_boxplot(bank["Months_on_book"])

• There are outliers on either side. Since these values lie close to the other values and some variation is expected, these will not be treated.

3.2.4 - Total_Relationship_Count

histogram_boxplot(bank["Total_Relationship_Count"])

• Can confirm above numbers: customers on average have approx. 4 products with bank (median - 4, min - 1, max - 6).

3.2.5 - Months_Inactive_12_mon

histogram_boxplot(bank["Months_Inactive_12_mon"])

• There are outliers on either side. Since these values are still close to the other values they will not be treated as outliers.

3.2.6 - Contacts_Count_12_mon

histogram_boxplot(bank["Contacts_Count_12_mon"])

• There are outliers on either side. Since these values are still close to the other values they will not be treated as outliers.

3.2.7 - Credit_Limit

histogram_boxplot(bank["Credit_Limit"])

• Distribution is right-skewed with outliers on the right. However, these will not be treated since some variation is expected in real-world variables like Credit Limit.

3.2.8 - Total_Revolving_Bal

histogram_boxplot(bank["Total_Revolving_Bal"])

• Many customers (nearly 2500) have a total revolving balance of \$0.

3.2.9 - Avg_Open_To_Buy

histogram_boxplot(bank["Avg_Open_To_Buy"])

• Distribution is right-skewed with outliers to the right. However, these will not be treated since some variation is expected in real-world variables like Average Open to Buy.

3.2.10 - Total_Trans_Amt

histogram_boxplot(bank["Total_Trans_Amt"])

• Distribution is right-skewed with outliers to the right. However, these will not be treated since some variation is expected in real-world variables like Total Transaction Amount.

3.2.11 - Total_Trans_Ct

histogram_boxplot(bank["Total_Trans_Ct"])

• Distribution is slightly right-skewed with outliers to the right. Need to investigate outlier values to determine if they will be treated.

# Checking 10 largest values of Total Transaction Count
bank.Total_Trans_Ct.nlargest(10)

9324     139
9586     138
9213     134
9629     132
9261     131
9269     131
9339     131
9728     131
9841     131
10085    131
Name: Total_Trans_Ct, dtype: int64

• Since outlier values are so close to other values, these values will not be treated.

3.2.12 - Total_Ct_Chng_Q4_Q1

histogram_boxplot(bank["Total_Ct_Chng_Q4_Q1"])

• Outliers exist on both the left and right sides. Outliers on the left need not be treated since they are close to the other values. Further investigation is needed to determine if outliers on the right should be treated.

bank[bank["Total_Ct_Chng_Q4_Q1"] > 1.5].sort_values(
    by="Total_Ct_Chng_Q4_Q1", ascending=True
)

	Attrition_Flag	Customer_Age	Gender	Dependent_count	Education_Level	Marital_Status	Income_Category	Card_Category	Months_on_book	Total_Relationship_Count	Months_Inactive_12_mon	Contacts_Count_12_mon	Credit_Limit	Total_Revolving_Bal	Avg_Open_To_Buy	Total_Amt_Chng_Q4_Q1	Total_Trans_Amt	Total_Trans_Ct	Total_Ct_Chng_Q4_Q1	Avg_Utilization_Ratio
4086	Existing Customer	50	M	3	College	Single	$40K - $60K	Blue	36	3	2	2	3069.000	1881	1188.000	1.111	2909	58	1.522	0.613
122	Existing Customer	53	M	1	High School	Married	$80K - $120K	Blue	47	3	2	2	12262.000	1836	10426.000	1.584	1659	38	1.533	0.150
284	Existing Customer	61	M	0	Graduate	Married	$40K - $60K	Blue	52	3	1	2	2939.000	1999	940.000	2.145	2434	33	1.538	0.680
132	Existing Customer	57	M	4	Graduate	Married	$80K - $120K	Blue	46	3	2	3	19270.000	1662	17608.000	1.186	1565	28	1.545	0.086
968	Existing Customer	54	M	2	Graduate	Married	$60K - $80K	Blue	42	6	2	0	3583.000	0	3583.000	1.088	1065	28	1.545	0.000
115	Existing Customer	49	M	1	Graduate	Single	$80K - $120K	Blue	36	6	2	2	18886.000	895	17991.000	1.826	1235	18	1.571	0.047
3221	Existing Customer	35	M	3	High School	NaN	$80K - $120K	Blue	30	5	3	3	11229.000	1054	10175.000	0.552	1898	36	1.571	0.094
22	Existing Customer	41	M	3	High School	Married	$40K - $60K	Blue	33	4	2	1	4470.000	680	3790.000	1.608	931	18	1.571	0.152
28	Existing Customer	44	F	3	Uneducated	Single	missing	Blue	34	5	2	2	10100.000	0	10100.000	0.525	1052	18	1.571	0.000
533	Existing Customer	59	F	0	NaN	Married	Less than $40K	Blue	53	4	3	2	5512.000	1124	4388.000	1.391	1975	49	1.579	0.204
7134	Existing Customer	46	M	3	High School	Married	$60K - $80K	Blue	36	4	3	1	4696.000	0	4696.000	0.732	4828	62	1.583	0.000
255	Existing Customer	55	F	2	High School	Married	Less than $40K	Blue	36	4	3	3	2358.000	1152	1206.000	0.969	1737	26	1.600	0.489
31	Existing Customer	53	M	2	Uneducated	Married	$60K - $80K	Blue	48	2	5	1	2451.000	1690	761.000	1.323	1596	26	1.600	0.690
2982	Existing Customer	45	M	4	NaN	Married	$40K - $60K	Blue	30	6	3	3	1892.000	1049	843.000	1.122	4558	60	1.609	0.554
251	Existing Customer	73	M	0	High School	Married	$40K - $60K	Blue	36	5	3	2	4469.000	1125	3344.000	1.363	1765	34	1.615	0.252
282	Existing Customer	54	F	1	Uneducated	Married	$40K - $60K	Blue	44	5	2	3	5556.000	1711	3845.000	1.434	1706	21	1.625	0.308
111	Existing Customer	42	M	3	NaN	Married	$60K - $80K	Blue	36	5	3	3	15088.000	865	14223.000	0.939	1107	21	1.625	0.057
17	Existing Customer	41	M	3	NaN	Married	$80K - $120K	Blue	34	4	4	1	13535.000	1291	12244.000	0.653	1028	21	1.625	0.095
0	Existing Customer	45	M	3	High School	Married	$60K - $80K	Blue	39	5	1	3	12691.000	777	11914.000	1.335	1144	42	1.625	0.061
392	Existing Customer	56	M	3	High School	Married	$60K - $80K	Blue	36	5	1	3	9226.000	1233	7993.000	1.521	1626	29	1.636	0.134
89	Existing Customer	57	M	2	NaN	Married	$120K +	Blue	45	5	3	3	5266.000	0	5266.000	1.702	1516	29	1.636	0.000
1187	Existing Customer	51	F	4	Graduate	Single	missing	Blue	34	3	1	2	11221.000	1279	9942.000	1.116	1564	45	1.647	0.114
2565	Existing Customer	39	M	3	Graduate	Married	$120K +	Blue	36	3	3	2	32964.000	2231	30733.000	1.731	3094	45	1.647	0.068
4597	Existing Customer	39	M	2	Graduate	Married	$60K - $80K	Blue	23	5	2	1	4761.000	1561	3200.000	0.690	2045	53	1.650	0.328
32	Existing Customer	41	M	4	Graduate	Married	$60K - $80K	Blue	36	4	1	2	8923.000	2517	6406.000	1.726	1589	24	1.667	0.282
281	Existing Customer	41	M	4	High School	NaN	$80K - $120K	Blue	36	4	3	0	23018.000	2168	20850.000	0.859	1463	24	1.667	0.094
9977	Attrited Customer	41	F	3	Uneducated	Married	Less than $40K	Blue	30	1	2	3	9491.000	0	9491.000	1.056	9061	51	1.684	0.000
47	Existing Customer	59	M	1	Doctorate	Married	$40K - $60K	Blue	52	3	2	2	2548.000	2020	528.000	2.357	1719	27	1.700	0.793
15	Existing Customer	44	M	4	NaN	NaN	$80K - $120K	Blue	37	5	1	2	4234.000	972	3262.000	1.707	1348	27	1.700	0.230
1778	Existing Customer	38	F	2	Uneducated	Married	Less than $40K	Blue	28	3	3	3	7049.000	0	7049.000	0.988	1475	27	1.700	0.000
2099	Existing Customer	65	F	1	Graduate	Single	missing	Blue	55	5	2	4	6463.000	1660	4803.000	0.995	2336	49	1.722	0.257
1041	Existing Customer	36	F	3	High School	Married	Less than $40K	Blue	26	6	1	4	4331.000	1445	2886.000	1.070	2053	33	1.750	0.334
322	Existing Customer	51	F	2	Uneducated	Divorced	Less than $40K	Blue	38	4	1	2	9648.000	1926	7722.000	0.549	1083	22	1.750	0.200
2683	Existing Customer	33	F	3	Post-Graduate	Married	$40K - $60K	Blue	28	5	2	3	1864.000	1081	783.000	1.416	2607	44	1.750	0.580
418	Existing Customer	44	M	5	Uneducated	Married	$60K - $80K	Blue	36	3	3	2	8645.000	1454	7191.000	0.708	1435	25	1.778	0.168
760	Existing Customer	62	F	0	NaN	Married	$40K - $60K	Blue	36	4	2	2	2964.000	2400	564.000	1.296	1557	25	1.778	0.810
4146	Existing Customer	48	M	4	Doctorate	NaN	$80K - $120K	Blue	40	3	2	2	3429.000	1268	2161.000	0.703	4422	64	1.783	0.370
88	Existing Customer	44	M	3	High School	Single	$60K - $80K	Blue	31	4	3	1	12756.000	837	11919.000	1.932	1413	14	1.800	0.066
346	Existing Customer	58	F	3	High School	Married	Less than $40K	Blue	47	6	1	0	2609.000	2439	170.000	1.269	1731	34	1.833	0.935
697	Existing Customer	55	M	2	High School	Married	$80K - $120K	Blue	36	3	2	3	6514.000	2258	4256.000	0.785	1383	20	1.857	0.347
76	Existing Customer	44	F	4	Graduate	Single	Less than $40K	Blue	36	6	4	2	7000.000	2517	4483.000	0.475	1112	23	1.875	0.360
324	Existing Customer	55	F	2	Uneducated	Married	missing	Blue	35	3	3	0	3025.000	2376	649.000	1.252	1277	23	1.875	0.785
323	Existing Customer	52	M	2	High School	Single	$80K - $120K	Silver	40	3	3	3	34516.000	2177	32339.000	1.320	1234	23	1.875	0.063
2358	Existing Customer	28	M	0	High School	Married	missing	Blue	21	5	3	5	25618.000	1209	24409.000	1.228	2157	49	1.882	0.047
151	Existing Customer	68	M	1	Graduate	Married	missing	Blue	56	5	2	3	13860.000	1652	12208.000	1.255	1910	32	1.909	0.119
52	Existing Customer	66	F	0	High School	Married	Less than $40K	Blue	54	3	4	2	3171.000	2179	992.000	1.224	1946	38	1.923	0.687
2696	Existing Customer	47	F	3	Uneducated	Married	Less than $40K	Blue	32	4	3	2	2821.000	951	1870.000	0.577	1452	38	1.923	0.337
300	Existing Customer	50	M	0	High School	Single	$120K +	Silver	45	5	2	3	14938.000	2303	12635.000	0.804	949	27	2.000	0.154
231	Existing Customer	57	M	2	NaN	Married	$80K - $120K	Blue	46	2	3	0	18871.000	1740	17131.000	1.727	1516	21	2.000	0.092
69	Existing Customer	50	M	2	Doctorate	Married	$80K - $120K	Blue	38	6	2	2	25300.000	1330	23970.000	1.072	837	15	2.000	0.053
1256	Existing Customer	43	M	2	High School	Divorced	$80K - $120K	Blue	33	4	3	2	3083.000	1898	1185.000	1.042	1448	30	2.000	0.616
13	Existing Customer	35	M	3	Graduate	NaN	$60K - $80K	Blue	30	5	1	3	8547.000	1666	6881.000	1.163	1311	33	2.000	0.195
456	Existing Customer	47	M	3	Uneducated	Married	$80K - $120K	Blue	36	6	2	2	29770.000	1565	28205.000	0.674	1774	42	2.000	0.053
84	Existing Customer	53	M	1	Graduate	Divorced	$80K - $120K	Blue	35	5	4	2	34516.000	1219	33297.000	1.129	1590	27	2.000	0.035
1455	Existing Customer	39	F	2	Doctorate	Married	missing	Blue	36	5	2	4	8058.000	791	7267.000	1.787	2742	42	2.000	0.098
294	Existing Customer	45	M	2	College	Single	$60K - $80K	Blue	33	6	3	0	23218.000	1814	21404.000	1.178	1749	37	2.083	0.078
309	Existing Customer	43	F	4	Uneducated	Married	Less than $40K	Blue	36	2	1	1	9684.000	2475	7209.000	0.697	1400	31	2.100	0.256
162	Existing Customer	46	M	1	Uneducated	Married	$60K - $80K	Blue	36	3	2	2	20459.000	2071	18388.000	1.248	1767	38	2.167	0.101
91	Existing Customer	49	M	4	High School	Single	$80K - $120K	Blue	38	4	3	0	31302.000	1953	29349.000	0.875	1564	35	2.182	0.062
131	Existing Customer	43	M	4	NaN	Married	$40K - $60K	Blue	39	5	1	2	6111.000	2517	3594.000	0.632	1221	16	2.200	0.412
757	Attrited Customer	43	M	2	Uneducated	Married	$60K - $80K	Blue	24	2	3	3	14447.000	0	14447.000	0.856	915	29	2.222	0.000
1095	Existing Customer	54	F	3	Doctorate	Married	$40K - $60K	Blue	39	3	1	3	2112.000	1594	518.000	1.204	1338	29	2.222	0.755
239	Existing Customer	44	M	4	NaN	Single	$80K - $120K	Blue	34	4	1	2	21573.000	1585	19988.000	0.621	1384	36	2.273	0.073
167	Existing Customer	58	F	4	Uneducated	Married	Less than $40K	Blue	47	3	3	3	2822.000	2173	649.000	1.456	1218	23	2.286	0.770
2	Existing Customer	51	M	3	Graduate	Married	$80K - $120K	Blue	36	4	1	0	3418.000	0	3418.000	2.594	1887	20	2.333	0.000
3	Existing Customer	40	F	4	High School	NaN	Less than $40K	Blue	34	3	4	1	3313.000	2517	796.000	1.405	1171	20	2.333	0.760
280	Existing Customer	43	M	1	Graduate	Single	$80K - $120K	Silver	37	4	3	2	34516.000	1440	33076.000	1.117	1575	34	2.400	0.042
68	Existing Customer	49	M	2	Graduate	Married	$60K - $80K	Blue	32	2	2	2	1687.000	1107	580.000	1.715	1670	17	2.400	0.656
158	Existing Customer	44	F	2	Uneducated	Married	missing	Silver	35	4	3	2	32643.000	0	32643.000	1.300	1058	24	2.429	0.000
4	Existing Customer	40	M	3	Uneducated	Married	$60K - $80K	Blue	21	5	1	0	4716.000	0	4716.000	2.175	816	28	2.500	0.000
805	Existing Customer	29	M	0	Post-Graduate	Single	$40K - $60K	Blue	19	4	1	2	13632.000	1482	12150.000	0.893	2168	42	2.500	0.109
2510	Attrited Customer	54	M	3	High School	Married	$40K - $60K	Blue	44	2	1	5	3032.000	0	3032.000	0.949	1037	14	2.500	0.000
30	Existing Customer	53	M	3	NaN	Married	$80K - $120K	Blue	33	3	2	3	2753.000	1811	942.000	0.977	1038	25	2.571	0.658
366	Existing Customer	36	F	4	Graduate	Married	$40K - $60K	Blue	36	6	3	3	1628.000	969	659.000	0.999	1893	15	2.750	0.595
146	Existing Customer	41	F	2	Graduate	Single	Less than $40K	Blue	32	6	3	2	2250.000	2117	133.000	1.162	1617	31	2.875	0.941
190	Existing Customer	57	M	1	Graduate	Married	$80K - $120K	Blue	47	5	3	1	14612.000	1976	12636.000	1.768	1827	24	3.000	0.135
113	Existing Customer	54	F	0	Uneducated	Married	Less than $40K	Blue	36	2	2	2	1494.000	706	788.000	1.674	1305	24	3.000	0.473
12	Existing Customer	56	M	1	College	Single	$80K - $120K	Blue	36	3	6	0	11751.000	0	11751.000	3.397	1539	17	3.250	0.000
269	Existing Customer	54	M	5	Graduate	Married	$60K - $80K	Blue	38	3	3	3	2290.000	1434	856.000	0.923	1119	18	3.500	0.626
773	Existing Customer	61	M	0	Post-Graduate	Married	missing	Blue	53	6	2	3	14434.000	1927	12507.000	2.675	1731	32	3.571	0.134
1	Existing Customer	49	F	5	Graduate	Single	Less than $40K	Blue	44	6	1	2	8256.000	864	7392.000	1.541	1291	33	3.714	0.105

• After examining the visualization together with the table of values of Total_Ct_Chng_Q4_Q1, the right outliers will be treated by capping them at 1.6.

bank["Total_Ct_Chng_Q4_Q1"].clip(upper=1.6, inplace=True)

3.2.13 - Total_Amt_Chng_Q4_Q1

histogram_boxplot(bank["Total_Amt_Chng_Q4_Q1"])

• Outliers exist on both the left and right. Outliers on the left need not be treated since they are close to the other values. Further investigation is needed to determine if outliers on the right should be treated.

bank[bank["Total_Amt_Chng_Q4_Q1"] > 1.5].sort_values(
    by="Total_Amt_Chng_Q4_Q1", ascending=True
)

	Attrition_Flag	Customer_Age	Gender	Dependent_count	Education_Level	Marital_Status	Income_Category	Card_Category	Months_on_book	Total_Relationship_Count	Months_Inactive_12_mon	Contacts_Count_12_mon	Credit_Limit	Total_Revolving_Bal	Avg_Open_To_Buy	Total_Amt_Chng_Q4_Q1	Total_Trans_Amt	Total_Trans_Ct	Total_Ct_Chng_Q4_Q1	Avg_Utilization_Ratio
358	Existing Customer	61	F	0	Graduate	Married	missing	Blue	56	4	1	3	5585.000	1465	4120.000	1.503	1787	31	0.722	0.262
2380	Existing Customer	35	M	1	High School	Married	$60K - $80K	Blue	26	4	3	2	4008.000	1841	2167.000	1.504	2794	58	1.148	0.459
1786	Existing Customer	33	F	2	High School	Married	Less than $40K	Blue	27	3	2	3	7185.000	0	7185.000	1.504	2679	45	0.800	0.000
1810	Existing Customer	37	M	3	Uneducated	Married	$80K - $120K	Blue	31	4	2	2	3797.000	1425	2372.000	1.505	2555	44	0.833	0.375
2012	Existing Customer	37	M	3	NaN	Married	$60K - $80K	Blue	30	6	2	4	16043.000	0	16043.000	1.509	2599	47	1.136	0.000
202	Existing Customer	48	M	3	College	Married	$80K - $120K	Blue	41	4	5	3	12302.000	1632	10670.000	1.509	1483	41	0.519	0.133
1990	Existing Customer	65	M	0	NaN	Single	missing	Blue	56	3	2	2	11332.000	633	10699.000	1.514	2758	56	1.240	0.056
1407	Existing Customer	33	F	3	Graduate	Married	Less than $40K	Blue	25	3	2	2	2307.000	1385	922.000	1.515	2440	48	0.846	0.600
656	Existing Customer	55	F	4	Graduate	Married	$40K - $60K	Blue	41	4	3	3	3156.000	2517	639.000	1.516	2091	48	0.778	0.798
495	Existing Customer	52	M	4	High School	Married	$80K - $120K	Blue	42	6	1	0	5738.000	1112	4626.000	1.519	1922	48	0.655	0.194
1521	Existing Customer	36	M	2	Doctorate	Married	$60K - $80K	Blue	30	3	3	4	4934.000	1504	3430.000	1.520	2747	42	0.448	0.305
392	Existing Customer	56	M	3	High School	Married	$60K - $80K	Blue	36	5	1	3	9226.000	1233	7993.000	1.521	1626	29	1.600	0.134
9	Existing Customer	48	M	2	Graduate	Single	$80K - $120K	Blue	36	6	3	3	11656.000	1677	9979.000	1.524	1441	32	0.882	0.144
1599	Existing Customer	35	F	1	High School	Married	missing	Blue	17	5	3	3	4588.000	2517	2071.000	1.525	2442	62	0.632	0.549
3061	Existing Customer	44	M	3	NaN	Married	$80K - $120K	Blue	38	3	1	4	4198.000	1782	2416.000	1.526	2051	36	1.000	0.424
263	Existing Customer	46	F	1	NaN	Married	Less than $40K	Blue	36	2	3	2	2474.000	2143	331.000	1.530	1541	40	1.000	0.866
1	Existing Customer	49	F	5	Graduate	Single	Less than $40K	Blue	44	6	1	2	8256.000	864	7392.000	1.541	1291	33	1.600	0.105
133	Existing Customer	52	M	1	Graduate	Married	$80K - $120K	Blue	43	3	1	4	3710.000	2517	1193.000	1.541	1578	34	0.700	0.678
919	Existing Customer	33	M	2	Post-Graduate	Married	$80K - $120K	Blue	22	3	3	3	1438.300	681	757.300	1.542	2730	48	0.778	0.473
1934	Existing Customer	36	F	2	NaN	Married	Less than $40K	Blue	27	3	3	4	3049.000	2198	851.000	1.546	2617	48	1.000	0.721
1060	Existing Customer	37	F	2	Graduate	Married	Less than $40K	Blue	30	5	2	2	2077.000	1392	685.000	1.553	1629	36	1.250	0.670
870	Existing Customer	53	M	3	Graduate	Married	$120K +	Blue	44	3	2	3	12163.000	2082	10081.000	1.554	2649	56	0.697	0.171
761	Existing Customer	58	M	0	Uneducated	Married	$80K - $120K	Blue	38	6	3	3	3785.000	1935	1850.000	1.558	1824	49	1.450	0.511
1647	Existing Customer	37	M	3	NaN	Married	$60K - $80K	Blue	24	3	1	2	11894.000	816	11078.000	1.561	2438	45	0.607	0.069
1011	Existing Customer	31	M	2	Uneducated	Married	Less than $40K	Blue	18	3	2	4	1438.300	0	1438.300	1.561	2789	50	1.083	0.000
1491	Existing Customer	36	F	3	High School	Married	Less than $40K	Blue	27	4	2	4	3444.000	2441	1003.000	1.563	2668	51	0.545	0.709
501	Existing Customer	55	M	1	High School	Married	$80K - $120K	Blue	45	3	3	3	16794.000	809	15985.000	1.566	1917	33	0.941	0.048
2761	Existing Customer	39	M	4	High School	Married	$80K - $120K	Blue	25	6	2	3	8208.000	0	8208.000	1.568	3135	39	1.167	0.000
906	Existing Customer	42	M	3	High School	Single	$40K - $60K	Blue	35	4	3	3	3266.000	1900	1366.000	1.568	2183	56	0.750	0.582
2752	Existing Customer	37	M	3	Graduate	Married	$60K - $80K	Blue	31	3	1	3	4940.000	1199	3741.000	1.570	3246	57	0.900	0.243
1882	Existing Customer	36	M	3	College	Married	$120K +	Blue	26	3	2	3	17116.000	939	16177.000	1.575	2547	45	0.731	0.055
1118	Existing Customer	29	M	0	High School	Married	$40K - $60K	Blue	13	4	1	4	1458.000	671	787.000	1.583	2449	56	0.931	0.460
122	Existing Customer	53	M	1	High School	Married	$80K - $120K	Blue	47	3	2	2	12262.000	1836	10426.000	1.584	1659	38	1.533	0.150
1051	Existing Customer	54	F	1	Uneducated	Married	Less than $40K	Blue	40	5	2	4	2178.000	1144	1034.000	1.585	1724	47	1.043	0.525
1195	Existing Customer	36	M	2	Graduate	Married	$80K - $120K	Blue	28	5	3	3	5634.000	1996	3638.000	1.589	2366	49	0.690	0.354
1008	Existing Customer	39	F	1	Graduate	Married	missing	Blue	30	3	2	3	2917.000	1357	1560.000	1.593	2422	46	0.438	0.465
2324	Existing Customer	37	F	4	Graduate	Married	Less than $40K	Blue	28	4	3	1	1825.000	1389	436.000	1.595	2878	54	0.636	0.761
1417	Existing Customer	39	F	1	Graduate	Married	$40K - $60K	Blue	27	3	1	2	2700.000	2220	480.000	1.595	2901	59	1.360	0.822
1166	Existing Customer	38	F	2	Graduate	Married	Less than $40K	Blue	36	6	3	4	2717.000	1591	1126.000	1.596	2064	33	1.062	0.586
1993	Existing Customer	31	M	2	College	Married	Less than $40K	Blue	21	6	2	4	2718.000	1920	798.000	1.604	2570	49	1.042	0.706
22	Existing Customer	41	M	3	High School	Married	$40K - $60K	Blue	33	4	2	1	4470.000	680	3790.000	1.608	931	18	1.571	0.152
2263	Existing Customer	45	F	4	Uneducated	Married	$40K - $60K	Blue	36	3	1	3	3111.000	2290	821.000	1.612	1998	41	0.464	0.736
45	Existing Customer	49	M	4	Uneducated	Single	$80K - $120K	Blue	30	3	2	3	34516.000	0	34516.000	1.621	1444	28	1.333	0.000
1473	Existing Customer	35	M	2	Graduate	Married	$60K - $80K	Blue	36	3	3	2	1438.300	0	1438.300	1.623	2492	55	0.897	0.000
2530	Existing Customer	35	M	3	Graduate	Married	$120K +	Blue	28	4	3	2	14869.000	0	14869.000	1.624	2753	48	1.087	0.000
1627	Existing Customer	38	F	2	Graduate	Married	Less than $40K	Blue	33	5	2	3	2228.000	1717	511.000	1.625	2334	61	0.848	0.771
1879	Existing Customer	35	F	1	Graduate	Married	$40K - $60K	Blue	29	4	2	2	2523.000	1877	646.000	1.631	2005	50	1.174	0.744
2321	Existing Customer	36	M	2	Uneducated	Married	$60K - $80K	Blue	21	4	1	5	12763.000	2517	10246.000	1.631	2599	46	0.917	0.197
914	Existing Customer	55	M	2	College	Married	$80K - $120K	Blue	45	5	1	2	2929.000	2431	498.000	1.632	1903	39	0.625	0.830
714	Existing Customer	38	M	1	High School	Married	$80K - $120K	Blue	23	4	3	2	22149.000	987	21162.000	1.633	2409	36	0.800	0.045
422	Existing Customer	63	M	0	High School	Married	$60K - $80K	Blue	36	6	2	3	2567.000	0	2567.000	1.636	2399	43	0.955	0.000
197	Existing Customer	53	M	3	Uneducated	Married	$80K - $120K	Blue	43	3	3	1	16393.000	1423	14970.000	1.642	1461	42	0.750	0.087
975	Existing Customer	59	M	1	Post-Graduate	Married	$40K - $60K	Blue	36	4	2	2	2483.000	1372	1111.000	1.642	1704	35	0.458	0.553
1232	Existing Customer	58	M	1	Uneducated	Married	$80K - $120K	Blue	50	5	3	2	6978.000	1148	5830.000	1.645	1362	26	1.000	0.165
95	Existing Customer	64	M	1	Graduate	Married	Less than $40K	Blue	52	6	4	3	1709.000	895	814.000	1.656	1673	32	0.882	0.524
1883	Existing Customer	37	M	2	College	Married	$80K - $120K	Blue	17	5	3	2	4631.000	1991	2640.000	1.669	2864	37	0.947	0.430
113	Existing Customer	54	F	0	Uneducated	Married	Less than $40K	Blue	36	2	2	2	1494.000	706	788.000	1.674	1305	24	1.600	0.473
3270	Existing Customer	49	M	3	High School	NaN	$60K - $80K	Blue	36	3	2	2	9551.000	1833	7718.000	1.675	3213	52	1.476	0.192
1570	Existing Customer	49	M	2	NaN	Single	$60K - $80K	Blue	38	4	1	2	2461.000	1586	875.000	1.676	1729	35	0.750	0.644
137	Existing Customer	45	M	4	College	Divorced	$60K - $80K	Blue	40	5	1	0	10408.000	1186	9222.000	1.689	2560	42	1.211	0.114
89	Existing Customer	57	M	2	NaN	Married	$120K +	Blue	45	5	3	3	5266.000	0	5266.000	1.702	1516	29	1.600	0.000
94	Existing Customer	45	F	3	NaN	Married	missing	Blue	28	5	1	2	2535.000	2440	95.000	1.705	1312	20	1.222	0.963
1689	Existing Customer	34	M	0	Graduate	Married	$60K - $80K	Blue	26	4	3	3	5175.000	977	4198.000	1.705	2405	49	0.885	0.189
15	Existing Customer	44	M	4	NaN	NaN	$80K - $120K	Blue	37	5	1	2	4234.000	972	3262.000	1.707	1348	27	1.600	0.230
336	Existing Customer	56	F	1	Graduate	Married	Less than $40K	Blue	38	4	3	3	2578.000	2462	116.000	1.707	1378	29	0.812	0.955
16	Existing Customer	48	M	4	Post-Graduate	Single	$80K - $120K	Blue	36	6	2	3	30367.000	2362	28005.000	1.708	1671	27	0.929	0.078
68	Existing Customer	49	M	2	Graduate	Married	$60K - $80K	Blue	32	2	2	2	1687.000	1107	580.000	1.715	1670	17	1.600	0.656
36	Existing Customer	55	F	3	Graduate	Married	Less than $40K	Blue	36	6	2	3	3035.000	2298	737.000	1.724	1877	37	1.176	0.757
32	Existing Customer	41	M	4	Graduate	Married	$60K - $80K	Blue	36	4	1	2	8923.000	2517	6406.000	1.726	1589	24	1.600	0.282
231	Existing Customer	57	M	2	NaN	Married	$80K - $120K	Blue	46	2	3	0	18871.000	1740	17131.000	1.727	1516	21	1.600	0.092
2565	Existing Customer	39	M	3	Graduate	Married	$120K +	Blue	36	3	3	2	32964.000	2231	30733.000	1.731	3094	45	1.600	0.068
2337	Existing Customer	50	F	2	Graduate	Divorced	$40K - $60K	Blue	40	6	2	5	8307.000	2517	5790.000	1.743	2293	36	0.800	0.303
1369	Existing Customer	36	F	2	Uneducated	Married	Less than $40K	Blue	36	4	2	2	4066.000	1639	2427.000	1.749	3040	56	0.931	0.403
33	Existing Customer	53	F	2	College	Married	Less than $40K	Blue	38	5	2	3	2650.000	1490	1160.000	1.750	1411	28	1.000	0.562
190	Existing Customer	57	M	1	Graduate	Married	$80K - $120K	Blue	47	5	3	1	14612.000	1976	12636.000	1.768	1827	24	1.600	0.135
1718	Existing Customer	42	F	4	Post-Graduate	Single	Less than $40K	Blue	36	6	2	3	1438.300	674	764.300	1.769	2451	55	1.292	0.469
1455	Existing Customer	39	F	2	Doctorate	Married	missing	Blue	36	5	2	4	8058.000	791	7267.000	1.787	2742	42	1.600	0.098
180	Existing Customer	45	M	2	Uneducated	Married	$40K - $60K	Blue	34	3	2	1	5771.000	2248	3523.000	1.791	1387	18	0.800	0.390
1486	Existing Customer	39	M	2	Graduate	Married	$40K - $60K	Blue	31	5	3	2	8687.000	1146	7541.000	1.800	2279	33	1.357	0.132
115	Existing Customer	49	M	1	Graduate	Single	$80K - $120K	Blue	36	6	2	2	18886.000	895	17991.000	1.826	1235	18	1.571	0.047
18	Existing Customer	61	M	1	High School	Married	$40K - $60K	Blue	56	2	2	3	3193.000	2517	676.000	1.831	1336	30	1.143	0.788
295	Existing Customer	60	M	0	High School	Married	$40K - $60K	Blue	36	5	1	3	3281.000	837	2444.000	1.859	1424	29	1.417	0.255
855	Existing Customer	39	F	2	Graduate	Married	missing	Blue	31	4	2	3	1438.300	997	441.300	1.867	2583	47	0.958	0.693
117	Existing Customer	50	M	3	High School	Single	$80K - $120K	Blue	39	4	1	4	9964.000	1559	8405.000	1.873	1626	25	0.786	0.156
1176	Existing Customer	34	M	2	College	Married	$80K - $120K	Blue	22	4	2	4	1631.000	0	1631.000	1.893	2962	57	1.111	0.000
869	Existing Customer	39	M	2	College	Married	$60K - $80K	Blue	35	4	3	2	7410.000	2517	4893.000	1.924	2398	37	1.176	0.340
88	Existing Customer	44	M	3	High School	Single	$60K - $80K	Blue	31	4	3	1	12756.000	837	11919.000	1.932	1413	14	1.600	0.066
6	Existing Customer	51	M	4	NaN	Married	$120K +	Gold	46	6	1	3	34516.000	2264	32252.000	1.975	1330	31	0.722	0.066
142	Existing Customer	54	M	4	Graduate	Married	$80K - $120K	Blue	34	2	3	2	14926.000	2517	12409.000	1.996	1576	25	1.500	0.169
431	Existing Customer	47	F	4	NaN	Divorced	$40K - $60K	Blue	34	6	1	2	3502.000	1851	1651.000	2.023	1814	31	0.722	0.529
1873	Existing Customer	38	M	3	Uneducated	Married	$60K - $80K	Blue	36	5	2	3	3421.000	2308	1113.000	2.037	2269	39	1.053	0.675
1085	Existing Customer	45	F	3	Graduate	Single	missing	Blue	36	3	3	4	11189.000	2517	8672.000	2.041	2959	58	1.231	0.225
177	Existing Customer	67	F	1	Graduate	Married	Less than $40K	Blue	56	4	3	2	3006.000	2517	489.000	2.053	1661	32	1.000	0.837
1219	Existing Customer	38	F	4	Graduate	Married	missing	Blue	28	4	1	2	6861.000	1598	5263.000	2.103	2228	39	0.950	0.233
154	Existing Customer	53	F	1	College	Married	Less than $40K	Blue	47	4	2	3	2154.000	930	1224.000	2.121	1439	26	1.364	0.432
284	Existing Customer	61	M	0	Graduate	Married	$40K - $60K	Blue	52	3	1	2	2939.000	1999	940.000	2.145	2434	33	1.538	0.680
4	Existing Customer	40	M	3	Uneducated	Married	$60K - $80K	Blue	21	5	1	0	4716.000	0	4716.000	2.175	816	28	1.600	0.000
841	Existing Customer	37	F	3	NaN	Married	Less than $40K	Blue	25	6	2	1	1438.300	674	764.300	2.180	1717	31	0.722	0.469
7	Existing Customer	32	M	0	High School	NaN	$60K - $80K	Silver	27	2	2	2	29081.000	1396	27685.000	2.204	1538	36	0.714	0.048
466	Existing Customer	63	M	2	Graduate	Married	$60K - $80K	Blue	49	5	2	3	14035.000	2061	11974.000	2.271	1606	30	1.500	0.147
58	Existing Customer	44	F	5	Graduate	Married	missing	Blue	35	4	1	2	6273.000	978	5295.000	2.275	1359	25	1.083	0.156
658	Existing Customer	46	M	4	Graduate	Married	$60K - $80K	Blue	35	5	1	2	1535.000	700	835.000	2.282	1848	25	1.083	0.456
46	Existing Customer	56	M	2	Doctorate	Married	$60K - $80K	Blue	45	6	2	0	2283.000	1430	853.000	2.316	1741	27	0.588	0.626
47	Existing Customer	59	M	1	Doctorate	Married	$40K - $60K	Blue	52	3	2	2	2548.000	2020	528.000	2.357	1719	27	1.600	0.793
219	Existing Customer	44	F	3	Uneducated	Divorced	Less than $40K	Silver	38	4	1	3	11127.000	1835	9292.000	2.368	1546	25	1.273	0.165
2	Existing Customer	51	M	3	Graduate	Married	$80K - $120K	Blue	36	4	1	0	3418.000	0	3418.000	2.594	1887	20	1.600	0.000
773	Existing Customer	61	M	0	Post-Graduate	Married	missing	Blue	53	6	2	3	14434.000	1927	12507.000	2.675	1731	32	1.600	0.134
8	Existing Customer	37	M	3	Uneducated	Single	$60K - $80K	Blue	36	5	2	0	22352.000	2517	19835.000	3.355	1350	24	1.182	0.113
12	Existing Customer	56	M	1	College	Single	$80K - $120K	Blue	36	3	6	0	11751.000	0	11751.000	3.397	1539	17	1.600	0.000

• After examining the visualization together with the table of values of Total_Amt_Chng_Q4_Q1, the right outliers will be treated by capping them at 1.75.

bank["Total_Amt_Chng_Q4_Q1"].clip(upper=1.75, inplace=True)

3.2.14 - Avg_Utilization_Ratio

histogram_boxplot(bank["Avg_Utilization_Ratio"])

• Approx. 2700 have an average utilization ratio of 0.

# Function that creates barplots with percentage displayed over each category.
# Learned/customized from debrief of Project 1 Cardio Good Fitness
def perc_on_bar(plot, feature):
    total = len(feature)  # length of the column
    for p in ax.patches:
        percentage = "{:.1f}%".format(
            100 * p.get_height() / total
        )  # percentage of each class of the category
        x = p.get_x() + p.get_width() / 2 - 0.05
        y = p.get_y() + p.get_height()
        ax.annotate(percentage, (x, y), size=18)
    plt.show()

3.2.15 - Attrition_Flag

plt.figure(figsize=(25, 15))
ax = sns.countplot(bank["Attrition_Flag"])
perc_on_bar(ax, bank["Attrition_Flag"])

• By a large margin, most customers are current customers.

3.2.16 - Gender

plt.figure(figsize=(25, 15))
ax = sns.countplot(bank["Gender"])
perc_on_bar(ax, bank["Gender"])

• By a small margin, most customers are female.

3.2.17 - Education_Level

plt.figure(figsize=(25, 15))
ax = sns.countplot(bank["Education_Level"])
perc_on_bar(ax, bank["Education_Level"])

• Most customers have a graduate degree.
• More customers are uneducated than have a college degree.
• Only a small percentage have doctorate degrees or post-graduate education.

3.2.18 - Marital_Status

plt.figure(figsize=(25, 15))
ax = sns.countplot(bank["Marital_Status"])
perc_on_bar(ax, bank["Marital_Status"])

• Most customers are married.
• Only a small percentage of customers are divorced.

3.2.19 - Income_Category

plt.figure(figsize=(25, 15))
ax = sns.countplot(bank["Income_Category"])
perc_on_bar(ax, bank["Income_Category"])

• By a large margin, most customers earn less than \$40,000.

3.2.20 - Card_Category

plt.figure(figsize=(25, 15))
ax = sns.countplot(bank["Card_Category"])
perc_on_bar(ax, bank["Card_Category"])

• By a significant margin, most customers are in the Blue card category.

3.3 - Bivariate Analysis

3.3.1 - Correlation

plt.figure(figsize=(20, 10))
sns.heatmap(bank.corr(), annot=True, vmin=-1, vmax=1, fmt=".2f")
plt.show()

The following are highly correlated:

• Months_on_book and Customer_Age (0.79)
• Credit_Limit and Avg_Open_To_Buy (1.00)
• Total_Trans_Amt and Total_Trans_Ct (0.81)

The following are moderately correlated:

• Credit_Limit and Avg_Utilization_Ratio (-0.48)
• Total_Revolving_Bal and Avg_Utilization_Ratio (0.62)
• Avg_Open_To_Buy and Avg_Utilization_Ratio (-0.54)

The following are slightly correlated:

• Total_Relationship_Count and Total_Trans_Amt (-0.35)
• Total_Relationship_Count and Total_Trans_Ct (-0.24)
• Total_Amt_Chng_Q4_Q1 and Total_Ct_Chng_Q4_Q1 (0.38)

sns.pairplot(bank, height=5, hue="Attrition_Flag")
plt.show()

3.3.2 - Customer_Age vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Customer_Age"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Customer_Age'>

• There seems to be no age difference in attrited customers vs. existing customers.

3.3.3 - Gender vs Attrition_Flag

### Function to plot stacked bar charts for categorical columns
### Code taken from Case Study and customized
def stacked_plot(x):
    sns.set(palette="nipy_spectral")
    tab1 = pd.crosstab(x, bank["Attrition_Flag"], margins=True)
    print(tab1)
    print("-" * 120)
    tab = pd.crosstab(x, bank["Attrition_Flag"], normalize="index")
    tab.plot(kind="bar", stacked=True, figsize=(10, 5))
    plt.legend(loc="lower left", frameon=False)
    plt.legend(loc="upper left", bbox_to_anchor=(1, 1))
    plt.show()

stacked_plot(bank["Gender"])

Attrition_Flag  Attrited Customer  Existing Customer    All
Gender                                                     
F                             930               4428   5358
M                             697               4072   4769
All                          1627               8500  10127
------------------------------------------------------------------------------------------------------------------------

• Number of males and females does not seem to differ with attrited customers vs. existing customers.

3.3.4 - Dependent_count vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Dependent_count"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Dependent_count'>

• There is a wider range of dependent count with existing customers.

3.3.5 - Education_Level vs Attrition_Flag

stacked_plot(bank["Education_Level"])

Attrition_Flag   Attrited Customer  Existing Customer   All
Education_Level                                            
College                        154                859  1013
Doctorate                       95                356   451
Graduate                       487               2641  3128
High School                    306               1707  2013
Post-Graduate                   92                424   516
Uneducated                     237               1250  1487
All                           1371               7237  8608
------------------------------------------------------------------------------------------------------------------------

• Among the attrited customers, slightly more are doctoral and post-graduates.

3.3.6 - Marital_Status vs Attrition_Flag

stacked_plot(bank["Marital_Status"])

Attrition_Flag  Attrited Customer  Existing Customer   All
Marital_Status                                            
Divorced                      121                627   748
Married                       709               3978  4687
Single                        668               3275  3943
All                          1498               7880  9378
------------------------------------------------------------------------------------------------------------------------

• There seems to be no difference in the categories when it comes to being an attrited vs an existing customer.

3.3.7 - Income_Category vs Attrition_Flag

stacked_plot(bank["Income_Category"])

Attrition_Flag   Attrited Customer  Existing Customer    All
Income_Category                                             
$120K +                        126                601    727
$40K - $60K                    271               1519   1790
$60K - $80K                    189               1213   1402
$80K - $120K                   242               1293   1535
Less than $40K                 612               2949   3561
missing                        187                925   1112
All                           1627               8500  10127
------------------------------------------------------------------------------------------------------------------------

• Slightly less in the 60K-80K category tend to be attrited.

3.3.8 - Card_Category vs Attrition_Flag

stacked_plot(bank["Card_Category"])

Attrition_Flag  Attrited Customer  Existing Customer    All
Card_Category                                              
Blue                         1519               7917   9436
Gold                           21                 95    116
Platinum                        5                 15     20
Silver                         82                473    555
All                          1627               8500  10127
------------------------------------------------------------------------------------------------------------------------

• Slightly larger number of Platinum members tend to be attrited.

3.3.9 - Months_on_book vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Months_on_book"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Months_on_book'>

• Existing customers tend to have been with the bank for a slightly wider range of time.

3.3.10 - Total_Relationship_Count vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Total_Relationship_Count"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Total_Relationship_Count'>

• Attrited customers include those who had less number of products with the bank.

3.3.11 - Months_Inactive_12_mon vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Months_Inactive_12_mon"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Months_Inactive_12_mon'>

• Existing customers include those who have been inactive less during the last 12-month period.

3.3.12 - Contacts_Count_12_mon vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Contacts_Count_12_mon"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Contacts_Count_12_mon'>

• Attrited customers tend to have had more contacts with the bank in the past 12 months.

3.3.13 - Credit_Limit vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Credit_Limit"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Credit_Limit'>

• Attrited customers have a slightly smaller credit limit than existing customers.

3.3.14 - Total_Revolving_Bal vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Total_Revolving_Bal"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Total_Revolving_Bal'>

• Existing customers have a higher revolving balance than attrited customers.

3.3.15 - Avg_Open_To_Buy vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Avg_Open_To_Buy"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Avg_Open_To_Buy'>

• Average Open to Buy is comparable between Attrited and Existing customers.

3.3.16 - Total_Trans_Amt vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Total_Trans_Amt"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Total_Trans_Amt'>

• Total Transaction Amount tends to be higher with existing customers.

3.3.17 - Total_Trans_Ct vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Total_Trans_Ct"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Total_Trans_Ct'>

• Total Transaction Count tends to be higher with existing customers.

3.3.18 - Total_Ct_Chng_Q4_Q1 vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Total_Ct_Chng_Q4_Q1"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Total_Ct_Chng_Q4_Q1'>

• The ratio tends to be lower with attrited customers.

3.3.19 - Total_Amt_Chng_Q4_Q1 vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Total_Amt_Chng_Q4_Q1"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Total_Amt_Chng_Q4_Q1'>

• The ratio has a wider range (range includes lower ratios) with attrited customers.

3.3.20 - Avg_Utilization_Ratio vs Attrition_Flag

plt.figure(figsize=(20, 10))
sns.boxplot(bank["Attrition_Flag"], bank["Avg_Utilization_Ratio"])

<AxesSubplot:xlabel='Attrition_Flag', ylabel='Avg_Utilization_Ratio'>

• Ratio range is wider (includes higher ratios) with existsing customers.

4 - Model Building - Data Preparation

4.1 - Split the Data

bank_copy = bank.copy()

X = bank_copy.drop(["Attrition_Flag"], axis=1)
y = bank_copy["Attrition_Flag"]
y = pd.get_dummies(data=y)
y.drop(["Existing Customer"], axis=1, inplace=True)

# Splitting data into training, validation and test sets:
# first we split data into 2 parts, say temporary and test

X_temp, X_test, y_temp, y_test = train_test_split(
    X, y, test_size=0.2, random_state=1, stratify=y
)

# then we split the temporary set into train and validation

X_train, X_val, y_train, y_val = train_test_split(
    X_temp, y_temp, test_size=0.25, random_state=1, stratify=y_temp
)
print(X_train.shape, X_val.shape, X_test.shape)

(6075, 19) (2026, 19) (2026, 19)

4.2 - Missing Value Treatment

imp_mode = SimpleImputer(missing_values=np.nan, strategy="most_frequent")

X_train["Education_Level"] = imp_mode.fit_transform(X_train[["Education_Level"]])
X_val["Education_Level"] = imp_mode.transform(X_val[["Education_Level"]])
X_test["Education_Level"] = imp_mode.transform(X_test[["Education_Level"]])

imp_mode2 = SimpleImputer(missing_values=np.nan, strategy="most_frequent")

X_train["Marital_Status"] = imp_mode2.fit_transform(X_train[["Marital_Status"]])
X_val["Marital_Status"] = imp_mode2.transform(X_val[["Marital_Status"]])
X_test["Marital_Status"] = imp_mode2.transform(X_test[["Marital_Status"]])

4.3 - Dummy Variables

# Creating dummy variables for categorical variables
X_train = pd.get_dummies(data=X_train, drop_first=True)
X_val = pd.get_dummies(data=X_val, drop_first=True)
X_test = pd.get_dummies(data=X_test, drop_first=True)

X_val.info()

<class 'pandas.core.frame.DataFrame'>
Int64Index: 2026 entries, 2894 to 6319
Data columns (total 30 columns):
 #   Column                          Non-Null Count  Dtype  
---  ------                          --------------  -----  
 0   Customer_Age                    2026 non-null   int64  
 1   Dependent_count                 2026 non-null   int64  
 2   Months_on_book                  2026 non-null   int64  
 3   Total_Relationship_Count        2026 non-null   int64  
 4   Months_Inactive_12_mon          2026 non-null   int64  
 5   Contacts_Count_12_mon           2026 non-null   int64  
 6   Credit_Limit                    2026 non-null   float64
 7   Total_Revolving_Bal             2026 non-null   int64  
 8   Avg_Open_To_Buy                 2026 non-null   float64
 9   Total_Amt_Chng_Q4_Q1            2026 non-null   float64
 10  Total_Trans_Amt                 2026 non-null   int64  
 11  Total_Trans_Ct                  2026 non-null   int64  
 12  Total_Ct_Chng_Q4_Q1             2026 non-null   float64
 13  Avg_Utilization_Ratio           2026 non-null   float64
 14  Gender_M                        2026 non-null   uint8  
 15  Education_Level_Doctorate       2026 non-null   uint8  
 16  Education_Level_Graduate        2026 non-null   uint8  
 17  Education_Level_High School     2026 non-null   uint8  
 18  Education_Level_Post-Graduate   2026 non-null   uint8  
 19  Education_Level_Uneducated      2026 non-null   uint8  
 20  Marital_Status_Married          2026 non-null   uint8  
 21  Marital_Status_Single           2026 non-null   uint8  
 22  Income_Category_$40K - $60K     2026 non-null   uint8  
 23  Income_Category_$60K - $80K     2026 non-null   uint8  
 24  Income_Category_$80K - $120K    2026 non-null   uint8  
 25  Income_Category_Less than $40K  2026 non-null   uint8  
 26  Income_Category_missing         2026 non-null   uint8  
 27  Card_Category_Gold              2026 non-null   uint8  
 28  Card_Category_Platinum          2026 non-null   uint8  
 29  Card_Category_Silver            2026 non-null   uint8  
dtypes: float64(5), int64(9), uint8(16)
memory usage: 269.1 KB

4.4 - Model Evaluation Criterion

Model can make the following wrong predictions:

1. Predicting customer attrited but they did not (False Positive)
2. Predicting that customer didn't attrite but they did (False Negative)

Which case is more important?

• Predicting that customer didn't attrite but they did (False Negative), i.e., losing a customer

How to reduce this loss, i.e., need to reduce False Negatives?

• The focus should be on maximizing Recall (minimizing False Negatives, identifying True Positives).

5 - Model Building and Evaluation

# The following functions were taken from a previous case study

# defining a function to compute different metrics to check performance of a classification model built using sklearn
def model_performance_classification_sklearn(model, predictors, target):
    """
    Function to compute different metrics to check classification model performance

    model: classifier
    predictors: independent variables
    target: dependent variable
    """

    # predicting using the independent variables
    pred = model.predict(predictors)

    acc = accuracy_score(target, pred)  # to compute Accuracy
    recall = recall_score(target, pred)  # to compute Recall
    precision = precision_score(target, pred)  # to compute Precision
    f1 = f1_score(target, pred)  # to compute F1-score

    # creating a dataframe of metrics
    df_perf = pd.DataFrame(
        {
            "Accuracy": acc,
            "Recall": recall,
            "Precision": precision,
            "F1": f1,
        },
        index=[0],
    )

    return df_perf

def confusion_matrix_sklearn(model, predictors, target):
    """
    To plot the confusion_matrix with percentages

    model: classifier
    predictors: independent variables
    target: dependent variable
    """
    y_pred = model.predict(predictors)
    cm = confusion_matrix(target, y_pred)
    labels = np.asarray(
        [
            ["{0:0.0f}".format(item) + "\n{0:.2%}".format(item / cm.flatten().sum())]
            for item in cm.flatten()
        ]
    ).reshape(2, 2)

    plt.figure(figsize=(6, 4))
    sns.heatmap(cm, annot=labels, fmt="")
    plt.ylabel("True label")
    plt.xlabel("Predicted label")

5.1 - Build 6 Models

models = []  # Empty list to store all the models

# Appending models into the list
models.append(("Logistic Regression", LogisticRegression(random_state=1)))
models.append(("Bagging", BaggingClassifier(random_state=1)))
models.append(("Random forest", RandomForestClassifier(random_state=1)))
models.append(("GBM", GradientBoostingClassifier(random_state=1)))
models.append(("Adaboost", AdaBoostClassifier(random_state=1)))
models.append(("dtree", DecisionTreeClassifier(random_state=1)))

results = []  # Empty list to store all model's CV scores
names = []  # Empty list to store name of the models


# loop through all models to get the mean cross validated score
print("\n" "Cross-Validation Performance:" "\n")

for name, model in models:
    scoring = "recall"
    kfold = StratifiedKFold(
        n_splits=5, shuffle=True, random_state=1
    )  # Setting number of splits equal to 5
    cv_result = cross_val_score(
        estimator=model, X=X_train, y=y_train, scoring=scoring, cv=kfold
    )
    results.append(cv_result)
    names.append(name)
    print("{}: {}".format(name, cv_result.mean() * 100))

print("\n" "Validation Performance:" "\n")

for name, model in models:
    model.fit(X_train, y_train)
    scores = recall_score(y_val, model.predict(X_val))
    print("{}: {}".format(name, scores))

Cross-Validation Performance:

Logistic Regression: 44.56776556776557
Bagging: 78.17425431711146
Random forest: 75.50863422291994
GBM: 81.3490319204605
Adaboost: 82.16745159602303
dtree: 78.79068550497121

Validation Performance:

Logistic Regression: 0.4754601226993865
Bagging: 0.8067484662576687
Random forest: 0.8128834355828221
GBM: 0.8588957055214724
Adaboost: 0.852760736196319
dtree: 0.8067484662576687

# Plotting boxplots for CV scores of all models defined above
fig = plt.figure()

fig.suptitle("Algorithm Comparison")
ax = fig.add_subplot(111)

plt.boxplot(results)
ax.set_xticklabels(names)

plt.show()

• On the training set, Adaboost has the highest performance followed by GBM, and dtree.
• On the validation set, GBM has the highest performance followed by Adaboost, and Random forest.
• The performance between validation and training differed less with dtree than Random forest.
• The models to be tuned are GBM, Adaboost, and dtree.

5.2 - RandomizedSearchCV

5.2.1 - Decision Tree

# defining model
model = DecisionTreeClassifier(random_state=1)

# Parameter grid to pass in RandomizedSearchCV
param_grid = {
    "criterion": ["gini", "entropy"],
    "max_depth": [3, 4, 5, None],
    "min_samples_split": [2, 4, 7, 10, 15],
}
# Type of scoring used to compare parameter combinations
scorer = metrics.make_scorer(metrics.recall_score)

# Calling RandomizedSearchCV
randomized_cv = RandomizedSearchCV(
    estimator=model,
    param_distributions=param_grid,
    n_iter=20,
    scoring=scorer,
    cv=5,
    random_state=1,
)

# Fitting parameters in RandomizedSearchCV
randomized_cv.fit(X_train, y_train)

print(
    "Best parameters are {} with CV score={}:".format(
        randomized_cv.best_params_, randomized_cv.best_score_
    )
)

Best parameters are {'min_samples_split': 2, 'max_depth': None, 'criterion': 'entropy'} with CV score=0.8032548403976977:

# Creating new pipeline with best parameters
dtree_tuned = DecisionTreeClassifier(
    random_state=1, criterion="entropy", max_depth=None, min_samples_split=2
)

# Fit the model on training data
dtree_tuned.fit(X_train, y_train)

DecisionTreeClassifier(criterion='entropy', random_state=1)

# Calculating different metrics on train set
dtree_random_train = model_performance_classification_sklearn(
    dtree_tuned, X_train, y_train
)
print("Training performance:")
dtree_random_train

Training performance:

	Accuracy	Recall	Precision	F1
0	1.000	1.000	1.000	1.000

# Calculating different metrics on validation set
dtree_random_val = model_performance_classification_sklearn(dtree_tuned, X_val, y_val)
print("Validation performance:")
dtree_random_val

Validation performance:

	Accuracy	Recall	Precision	F1
0	0.931	0.776	0.791	0.783

# creating confusion matrix
confusion_matrix_sklearn(dtree_tuned, X_val, y_val)

• There is overfitting on the training data.
• 0.81 was the cross-validated recall score and the recall score on the validated data here is 0.78 so slight drop.

5.2.2 - Adaboost

# defining model
model = AdaBoostClassifier(random_state=1)

# Parameter grid to pass in GridSearchCV

param_grid = {
    "n_estimators": np.arange(10, 110, 10),
    "learning_rate": [0.1, 0.01, 0.2, 0.05, 1],
    "base_estimator": [
        DecisionTreeClassifier(max_depth=1, random_state=1),
        DecisionTreeClassifier(max_depth=2, random_state=1),
        DecisionTreeClassifier(max_depth=3, random_state=1),
    ],
}

# Type of scoring used to compare parameter combinations
scorer = metrics.make_scorer(metrics.recall_score)

# Calling RandomizedSearchCV
randomized_cv = RandomizedSearchCV(
    estimator=model,
    param_distributions=param_grid,
    n_jobs=-1,
    n_iter=50,
    scoring=scorer,
    cv=5,
    random_state=1,
)

# Fitting parameters in RandomizedSearchCV
randomized_cv.fit(X_train, y_train)

print(
    "Best parameters are {} with CV score={}:".format(
        randomized_cv.best_params_, randomized_cv.best_score_
    )
)

Best parameters are {'n_estimators': 100, 'learning_rate': 1, 'base_estimator': DecisionTreeClassifier(max_depth=2, random_state=1)} with CV score=0.8596127681841967:

# building model with best parameters
adb_tuned = AdaBoostClassifier(
    n_estimators=100,
    learning_rate=1,
    random_state=1,
    base_estimator=DecisionTreeClassifier(max_depth=2, random_state=1),
)

# Fit the model on training data
adb_tuned.fit(X_train, y_train)

AdaBoostClassifier(base_estimator=DecisionTreeClassifier(max_depth=2,
                                                         random_state=1),
                   learning_rate=1, n_estimators=100, random_state=1)

# Calculating different metrics on train set
Adaboost_random_train = model_performance_classification_sklearn(
    adb_tuned, X_train, y_train
)
print("Training performance:")
Adaboost_random_train

Training performance:

	Accuracy	Recall	Precision	F1
0	0.998	0.991	0.994	0.992

# Calculating different metrics on validation set
Adaboost_random_val = model_performance_classification_sklearn(adb_tuned, X_val, y_val)
print("Validation performance:")
Adaboost_random_val

Validation performance:

	Accuracy	Recall	Precision	F1
0	0.966	0.865	0.919	0.891

# creating confusion matrix
confusion_matrix_sklearn(adb_tuned, X_val, y_val)

• There seems to be overfitting on the training set.
• Cross-validation recall score was 0.85 and recall score here is 0.86 so slight improvement.

5.2.3 - GBM

# defining model
model = GradientBoostingClassifier(random_state=1)

# Parameter grid to pass in GridSearchCV

param_grid = {
    "n_estimators": np.arange(10, 110, 10),
    "learning_rate": [0.1, 0.01, 0.2, 0.05, 1],
}

# Type of scoring used to compare parameter combinations
scorer = metrics.make_scorer(metrics.recall_score)

# Calling RandomizedSearchCV
randomized_cv = RandomizedSearchCV(
    estimator=model,
    param_distributions=param_grid,
    n_jobs=-1,
    n_iter=50,
    scoring=scorer,
    cv=5,
    random_state=1,
)

# Fitting parameters in RandomizedSearchCV
randomized_cv.fit(X_train, y_train)

print(
    "Best parameters are {} with CV score={}:".format(
        randomized_cv.best_params_, randomized_cv.best_score_
    )
)

Best parameters are {'n_estimators': 100, 'learning_rate': 0.2} with CV score=0.8473103087388802:

# building model with best parameters
gbm_tuned = GradientBoostingClassifier(
    n_estimators=100,
    learning_rate=0.2,
    random_state=1,
)

# Fit the model on training data
gbm_tuned.fit(X_train, y_train)

GradientBoostingClassifier(learning_rate=0.2, random_state=1)

# Calculating different metrics on train set
gbm_random_train = model_performance_classification_sklearn(gbm_tuned, X_train, y_train)
print("Training performance:")
gbm_random_train

Training performance:

	Accuracy	Recall	Precision	F1
0	0.986	0.942	0.969	0.955

# Calculating different metrics on validation set
gbm_random_val = model_performance_classification_sklearn(gbm_tuned, X_val, y_val)
print("Validation performance:")
gbm_random_val

Validation performance:

	Accuracy	Recall	Precision	F1
0	0.970	0.877	0.935	0.905

# creating confusion matrix
confusion_matrix_sklearn(gbm_tuned, X_val, y_val)

• Slight improvement from performance on cross-validation (0.86) on validation set here is 0.88.
• Model seems to perform well on training and validation set.

5.3 - Oversampling train data using SMOTE

print(
    "Before UpSampling, counts of label 'Yes': {}".format(
        sum(y_train["Attrited Customer"] == 1)
    )
)
print(
    "Before UpSampling, counts of label 'No': {} \n".format(
        sum(y_train["Attrited Customer"] == 0)
    )
)

sm = SMOTE(
    sampling_strategy=1, k_neighbors=5, random_state=1
)  # Synthetic Minority Over Sampling Technique
X_train_over, y_train_over = sm.fit_resample(X_train, y_train)


print(
    "After UpSampling, counts of label 'Yes': {}".format(
        sum(y_train_over["Attrited Customer"] == 1)
    )
)
print(
    "After UpSampling, counts of label 'No': {} \n".format(
        sum(y_train_over["Attrited Customer"] == 0)
    )
)


print("After UpSampling, the shape of train_X: {}".format(X_train_over.shape))
print("After UpSampling, the shape of train_y: {} \n".format(y_train_over.shape))

Before UpSampling, counts of label 'Yes': 976
Before UpSampling, counts of label 'No': 5099 

After UpSampling, counts of label 'Yes': 5099
After UpSampling, counts of label 'No': 5099 

After UpSampling, the shape of train_X: (10198, 30)
After UpSampling, the shape of train_y: (10198, 1) 

5.3.1 - Decision Tree on oversampled data

dt_over = DecisionTreeClassifier(random_state=1)

dt_over.fit(X_train_over, y_train_over)

DecisionTreeClassifier(random_state=1)

Performance Evaluation Using KFold and cross_val_score

scoring = "recall"
kfold = StratifiedKFold(
    n_splits=5, shuffle=True, random_state=1
)  # Setting number of splits equal to 5
dt_cv_result_over = cross_val_score(
    estimator=dt_over, X=X_train_over, y=y_train_over, scoring=scoring, cv=kfold
)
# Plotting boxplots for CV scores of model defined above
plt.boxplot(dt_cv_result_over)
plt.show()

# Calculating different metrics on train set
dt_over_train_perf = model_performance_classification_sklearn(
    dt_over, X_train_over, y_train_over
)
print("Training performance:")
dt_over_train_perf

Training performance:

	Accuracy	Recall	Precision	F1
0	1.000	1.000	1.000	1.000

• There is overfitting on training test set.

# Calculating different metrics on validation set
dt_over_val_perf = model_performance_classification_sklearn(dt_over, X_val, y_val)
print("Validation performance:")
dt_over_val_perf

Validation performance:

	Accuracy	Recall	Precision	F1
0	0.932	0.847	0.758	0.800

• This is an improvement over the dtree_tuned model above which had recall score of 0.78 for the validation set.

# creating confusion matrix
confusion_matrix_sklearn(dt_over, X_val, y_val)

• Overall, the model is still overfitting for the training set (just as it did for tuned model above) and has slight improvement over the tuned model in terms of recall for the validation set compared to the tuned model above.

5.3.2 - AdaBoost on oversampled data

adb_over = AdaBoostClassifier(random_state=1)

adb_over.fit(X_train_over, y_train_over)

AdaBoostClassifier(random_state=1)

Performance Evaluation Using KFold and cross_val_score

scoring = "recall"
kfold = StratifiedKFold(
    n_splits=5, shuffle=True, random_state=1
)  # Setting number of splits equal to 5
adb_cv_result_over = cross_val_score(
    estimator=adb_over, X=X_train_over, y=y_train_over, scoring=scoring, cv=kfold
)
# Plotting boxplots for CV scores of model defined above
plt.boxplot(adb_cv_result_over)
plt.show()

# Calculating different metrics on train set
adb_over_train_perf = model_performance_classification_sklearn(
    adb_over, X_train_over, y_train_over
)
print("Training performance:")
adb_over_train_perf

Training performance:

	Accuracy	Recall	Precision	F1
0	0.962	0.967	0.958	0.962

• There seems to be slightly less overfitting here than in previous tuned model.

# Calculating different metrics on validation set
adb_over_val_perf = model_performance_classification_sklearn(adb_over, X_val, y_val)
print("validation performance:")
adb_over_val_perf

validation performance:

	Accuracy	Recall	Precision	F1
0	0.945	0.883	0.796	0.837

• In terms of recall, this model is performing better than the previous tuned model.

# creating confusion matrix
confusion_matrix_sklearn(adb_over, X_val, y_val)

• There is still some overfitting when comparing training to validation set, but in terms of recall, this model is performing better than previously tuned model.

5.3.3 - GMB on oversampled data

gbm_over = GradientBoostingClassifier(random_state=1)

gbm_over.fit(X_train_over, y_train_over)

GradientBoostingClassifier(random_state=1)

Performance Evaluation Using KFold and cross_val_score

scoring = "recall"
kfold = StratifiedKFold(
    n_splits=5, shuffle=True, random_state=1
)  # Setting number of splits equal to 5
gbm_cv_result_over = cross_val_score(
    estimator=gbm_over, X=X_train_over, y=y_train_over, scoring=scoring, cv=kfold
)
# Plotting boxplots for CV scores of model defined above
plt.boxplot(gbm_cv_result_over)
plt.show()

# Calculating different metrics on train set
gbm_over_train_perf = model_performance_classification_sklearn(
    gbm_over, X_train_over, y_train_over
)
print("Training performance:")
gbm_over_train_perf

Training performance:

	Accuracy	Recall	Precision	F1
0	0.977	0.981	0.973	0.977

• Results are largely comparable with tuned model, although in terms of recall this model performs slightly higher on the training set.

# Calculating different metrics on validation set
gbm_over_val_perf = model_performance_classification_sklearn(gbm_over, X_val, y_val)
print("validation performance:")
gbm_over_val_perf

validation performance:

	Accuracy	Recall	Precision	F1
0	0.957	0.893	0.846	0.869

• Only in terms of recall does this model perform better on validation set than the tuned model.

# creating confusion matrix
confusion_matrix_sklearn(gbm_over, X_val, y_val)

• In terms of recall, this model performs better than the tuned model.

5.4 - Undersampling train data using Random Under Sampler

rus = RandomUnderSampler(random_state=1)
X_train_un, y_train_un = rus.fit_resample(X_train, y_train)

print(
    "Before Under Sampling, counts of label 'Yes': {}".format(
        sum(y_train["Attrited Customer"] == 1)
    )
)
print(
    "Before Under Sampling, counts of label 'No': {} \n".format(
        sum(y_train["Attrited Customer"] == 0)
    )
)

print(
    "After Under Sampling, counts of label 'Yes': {}".format(
        sum(y_train_un["Attrited Customer"] == 1)
    )
)
print(
    "After Under Sampling, counts of label 'No': {} \n".format(
        sum(y_train_un["Attrited Customer"] == 0)
    )
)

print("After Under Sampling, the shape of train_X: {}".format(X_train_un.shape))
print("After Under Sampling, the shape of train_y: {} \n".format(y_train_un.shape))

Before Under Sampling, counts of label 'Yes': 976
Before Under Sampling, counts of label 'No': 5099 

After Under Sampling, counts of label 'Yes': 976
After Under Sampling, counts of label 'No': 976 

After Under Sampling, the shape of train_X: (1952, 30)
After Under Sampling, the shape of train_y: (1952, 1) 

5.4.1 - Decision Tree on undersampled data

dt_under = DecisionTreeClassifier(random_state=1)
dt_under.fit(X_train_un, y_train_un)

DecisionTreeClassifier(random_state=1)

Performance Evaluation Using KFold and cross_val_score

scoring = "recall"
kfold = StratifiedKFold(
    n_splits=5, shuffle=True, random_state=1
)  # Setting number of splits equal to 5
dt_cv_result_under = cross_val_score(
    estimator=dt_under, X=X_train_un, y=y_train_un, scoring=scoring, cv=kfold
)
# Plotting boxplots for CV scores of model defined above
plt.boxplot(dt_cv_result_under)
plt.show()

# Calculating different metrics on train set
dt_under_train_perf = model_performance_classification_sklearn(
    dt_under, X_train_un, y_train_un
)
print("Training performance:")
dt_under_train_perf

Training performance:

	Accuracy	Recall	Precision	F1
0	1.000	1.000	1.000	1.000

• Just as with the prior 2 models, there is overfitting on the training set with this model.

# Calculating different metrics on validation set
dt_under_val_perf = model_performance_classification_sklearn(dt_under, X_val, y_val)
print("Validation performance:")
dt_under_val_perf

Validation performance:

	Accuracy	Recall	Precision	F1
0	0.891	0.896	0.611	0.726

• In terms of recall, this model is an improvement over both the tuned model and the oversampled model on the validation set.

# creating confusion matrix
confusion_matrix_sklearn(dt_under, X_val, y_val)

• There is an overfitting problem between training and validation sets, but compared to the previous 2 models in terms of recall, this model performs better.

5.4.2 - AdaBoost on undersampled data

adb_under = AdaBoostClassifier(random_state=1)
adb_under.fit(X_train_un, y_train_un)

AdaBoostClassifier(random_state=1)

Performance Evaluation Using KFold and cross_val_score

scoring = "recall"
kfold = StratifiedKFold(
    n_splits=5, shuffle=True, random_state=1
)  # Setting number of splits equal to 5
adb_cv_result_under = cross_val_score(
    estimator=adb_under, X=X_train_un, y=y_train_un, scoring=scoring, cv=kfold
)
# Plotting boxplots for CV scores of model defined above
plt.boxplot(adb_cv_result_under)
plt.show()

# Calculating different metrics on train set
adb_under_train_perf = model_performance_classification_sklearn(
    adb_under, X_train_un, y_train_un
)
print("Training performance:")
adb_under_train_perf

Training performance:

	Accuracy	Recall	Precision	F1
0	0.950	0.953	0.947	0.950

• On the testing set, this model does not perform as well as the previous 2 models.

# Calculating different metrics on validation set
adb_under_val_perf = model_performance_classification_sklearn(adb_under, X_val, y_val)
print("Validation performance:")
adb_under_val_perf

Validation performance:

	Accuracy	Recall	Precision	F1
0	0.929	0.960	0.705	0.813

• In terms of recall, on the validation set, this model performs better than the previous 2 models.

# creating confusion matrix
confusion_matrix_sklearn(adb_under, X_val, y_val)

• There is a loss of performance on the testing set comparing this model to the 2 previous models, but the model is an improvement in terms of recall on the validation set compared to the 2 previous models.

5.4.2 - GBM on undersampled data

gbm_under = DecisionTreeClassifier(random_state=1)
gbm_under.fit(X_train_un, y_train_un)

DecisionTreeClassifier(random_state=1)

Performance Evaluation Using KFold and cross_val_score

scoring = "recall"
kfold = StratifiedKFold(
    n_splits=5, shuffle=True, random_state=1
)  # Setting number of splits equal to 5
gbm_cv_result_under = cross_val_score(
    estimator=gbm_under, X=X_train_un, y=y_train_un, scoring=scoring, cv=kfold
)
# Plotting boxplots for CV scores of model defined above
plt.boxplot(gbm_cv_result_under)
plt.show()

# Calculating different metrics on train set
gbm_under_train_perf = model_performance_classification_sklearn(
    gbm_under, X_train_un, y_train_un
)
print("Training performance:")
gbm_under_train_perf

Training performance:

	Accuracy	Recall	Precision	F1
0	1.000	1.000	1.000	1.000

• Just as with the prevous 2 models, this model is overfitting on the training set.

# Calculating different metrics on validation set
gbm_under_val_perf = model_performance_classification_sklearn(gbm_under, X_val, y_val)
print("Validation performance:")
gbm_under_val_perf

Validation performance:

	Accuracy	Recall	Precision	F1
0	0.891	0.896	0.611	0.726

• Overall, this model performs worse on the validation set than the previous 2 models.

# creating confusion matrix
confusion_matrix_sklearn(gbm_under, X_val, y_val)

• This model suffers from overfitting on the training set and performs poorly on all metrics on the validation set compared to the other 2 GBM models.

6 - Comparing All Models

# training performance comparison

models_train_comp_df = pd.concat(
    [
        dtree_random_train.T,
        Adaboost_random_train.T,
        gbm_random_train.T,
        dt_over_train_perf.T,
        adb_over_train_perf.T,
        gbm_over_train_perf.T,
        dt_under_train_perf.T,
        adb_under_train_perf.T,
        gbm_under_train_perf.T,
    ],
    axis=1,
)
models_train_comp_df.columns = [
    "DTree Tuned",
    "ADB Tuned",
    "GBM Tuned",
    "DTree Over",
    "ADB Over",
    "GBM Over",
    "DTree Under",
    "ADB Under",
    "GBM Under",
]

print("Training Performance Comparison")
models_train_comp_df

Training Performance Comparison

	DTree Tuned	ADB Tuned	GBM Tuned	DTree Over	ADB Over	GBM Over	DTree Under	ADB Under	GBM Under
Accuracy	1.000	0.998	0.986	1.000	0.962	0.977	1.000	0.950	1.000
Recall	1.000	0.991	0.942	1.000	0.967	0.981	1.000	0.953	1.000
Precision	1.000	0.994	0.969	1.000	0.958	0.973	1.000	0.947	1.000
F1	1.000	0.992	0.955	1.000	0.962	0.977	1.000	0.950	1.000

# Validation performance comparison

models_val_comp_df = pd.concat(
    [
        dtree_random_val.T,
        Adaboost_random_val.T,
        gbm_random_val.T,
        dt_over_val_perf.T,
        adb_over_val_perf.T,
        gbm_over_val_perf.T,
        dt_under_val_perf.T,
        adb_under_val_perf.T,
        gbm_under_val_perf.T,
    ],
    axis=1,
)
models_val_comp_df.columns = [
    "DTree Tuned",
    "ADB Tuned",
    "GBM Tuned",
    "DTree Over",
    "ADB Over",
    "GBM Over",
    "DTree Under",
    "ADB Under",
    "GBM Under",
]

print("Validation Performance Comparison")
models_val_comp_df

Validation Performance Comparison

	DTree Tuned	ADB Tuned	GBM Tuned	DTree Over	ADB Over	GBM Over	DTree Under	ADB Under	GBM Under
Accuracy	0.931	0.966	0.970	0.932	0.945	0.957	0.891	0.929	0.891
Recall	0.776	0.865	0.877	0.847	0.883	0.893	0.896	0.960	0.896
Precision	0.791	0.919	0.935	0.758	0.796	0.846	0.611	0.705	0.611
F1	0.783	0.891	0.905	0.800	0.837	0.869	0.726	0.813	0.726

• AdaBoost on the Undersampled Data gives the highest recall score on the validation set. In addition, it scores similarly on the training set which seems to point to less of an overfitting problem. Therefore, this is deemed the best model.

# Feature Importance of Adaboost Model on Undersampled Data
feature_names = X_train.columns
importances = adb_under.feature_importances_
indices = np.argsort(importances)

plt.figure(figsize=(12, 12))
plt.title("Feature Importances")
plt.barh(range(len(indices)), importances[indices], color="violet", align="center")
plt.yticks(range(len(indices)), [feature_names[i] for i in indices])
plt.xlabel("Relative Importance")
plt.show()

• Total Transaction Amount is by far the most important variable.
• Total Transaction Count is the second most important variable.
• Total Revolving Balance and Total_Ct_Chng_Q4_Q1 ratio are tied for third most important variable.

Performance on the Test Set

# Calculating different metrics on validation set
adb_under_test_perf = model_performance_classification_sklearn(
    adb_under, X_test, y_test
)
print("Test performance:")
adb_under_test_perf

Test performance:

	Accuracy	Recall	Precision	F1
0	0.928	0.960	0.701	0.810

7 - Pipelines for Productionizing the Model

Creating 2 pipelines - one will be for numerical columns (missing value imputation) and one will be for categorical columns (one hot encoding and missing value imputation).

# creating a list of numerical variables
numerical_features = [
    "Customer_Age",
    "Dependent_count",
    "Months_on_book",
    "Total_Relationship_Count",
    "Months_Inactive_12_mon",
    "Contacts_Count_12_mon",
    "Credit_Limit",
    "Total_Revolving_Bal",
    "Avg_Open_To_Buy",
    "Total_Trans_Amt",
    "Total_Trans_Ct",
    "Total_Ct_Chng_Q4_Q1",
    "Total_Amt_Chng_Q4_Q1",
    "Avg_Utilization_Ratio",
]

# creating a transformer for numerical variables, which will apply simple imputer on the numerical variables
numeric_transformer = Pipeline(steps=[("imputer", SimpleImputer(strategy="median"))])

# creating a list of categorical variables
categorical_features = [
    "Gender",
    "Education_Level",
    "Marital_Status",
    "Income_Category",
    "Card_Category",
]

# creating a transformer for categorical variables, which will first apply simple imputer and
# then do one hot encoding for categorical variables
categorical_transformer = Pipeline(
    steps=[
        ("imputer", SimpleImputer(strategy="most_frequent")),
        ("onehot", OneHotEncoder(handle_unknown="ignore")),
    ]
)

# handle_unknown = "ignore", allows model to handle any unknown category in the test data

# combining categorical transformer and numerical transformer using a column transformer

preprocessor = ColumnTransformer(
    transformers=[
        ("num", numeric_transformer, numerical_features),
        ("cat", categorical_transformer, categorical_features),
    ],
    remainder="passthrough",
)
# remainder = "passthrough" has been used, it will allow variables that are present in original data
# but not in "numerical_columns" and "categorical_columns" to pass through the column transformer without any changes

# Separating target variable and other variables
X = bank.drop(["Attrition_Flag"], axis=1)
Y = bank["Attrition_Flag"]
Y = pd.get_dummies(data=Y)
Y.drop(["Existing Customer"], axis=1, inplace=True)

Y

	Attrited Customer
0	0
1	0
2	0
3	0
4	0
...	...
10122	0
10123	1
10124	1
10125	1
10126	1

10127 rows × 1 columns

# Splitting the data into train and test sets
X_train, X_test, y_train, y_test = train_test_split(
    X, Y, test_size=0.30, random_state=1, stratify=Y
)
print(X_train.shape, X_test.shape)

(7088, 19) (3039, 19)

# Creating new pipeline with best parameters
model = Pipeline(
    steps=[
        ("pre", preprocessor),
        (
            "ADB Undersampled",
            AdaBoostClassifier(random_state=1),
        ),
    ]
)

# Undersample the data
rus = RandomUnderSampler(random_state=1)
X_train_un, y_train_un = rus.fit_resample(X_train, y_train)

# Fit the model on undersampled training data
model.fit(X_train_un, y_train_un)

Pipeline(steps=[('pre',
                 ColumnTransformer(remainder='passthrough',
                                   transformers=[('num',
                                                  Pipeline(steps=[('imputer',
                                                                   SimpleImputer(strategy='median'))]),
                                                  ['Customer_Age',
                                                   'Dependent_count',
                                                   'Months_on_book',
                                                   'Total_Relationship_Count',
                                                   'Months_Inactive_12_mon',
                                                   'Contacts_Count_12_mon',
                                                   'Credit_Limit',
                                                   'Total_Revolving_Bal',
                                                   'Avg_Open_To_Buy',
                                                   'Total_Trans_Amt',
                                                   '...l_Trans_Ct',
                                                   'Total_Ct_Chng_Q4_Q1',
                                                   'Total_Amt_Chng_Q4_Q1',
                                                   'Avg_Utilization_Ratio']),
                                                 ('cat',
                                                  Pipeline(steps=[('imputer',
                                                                   SimpleImputer(strategy='most_frequent')),
                                                                  ('onehot',
                                                                   OneHotEncoder(handle_unknown='ignore'))]),
                                                  ['Gender', 'Education_Level',
                                                   'Marital_Status',
                                                   'Income_Category',
                                                   'Card_Category'])])),
                ('ADB Undersampled', AdaBoostClassifier(random_state=1))])

8 - Business Recommendations

• Total Transaction Amount is the most important variable in terms of determining if a customer is attrited. This value tends to be higher for existing customers. Thera Bank in an effort to increase balances could offer programs for credit card holders where they earn points for dollars spent, incentivizing spending with the credit card. Similarly, they could create a balance transfer offer (0% or low percent interest rate on balance tranfers for an allotted period of time).
• Total_Trans_Ct or Total Transaction Count is also an important variable in determining if a customer is attrited. This number tends to be higher with existing customers. Thera Bank could investigate offering deals at certain merchants if their card(s) are used (e.g., discount when using your Thera Bank credit or debit card) incentivizing the increase in transactions.
• Most attrited customers make less than \$40,000. Thera Bank could target this population by offering lower interest rates on cards with smaller credit limits, or cards with lower fee schedules.
• Attrited customers tend to have had less products with the bank. Thera Bank could research offering new membership plans or accounts to people with only 1 card (e.g., no fees or lower interest rates if you have both a debit and credit card with the bank).